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Overview 


The basic structure of this book is simple to understand. It covers optimization 
methods and applications in discrete time and in continuous time, both in worlds with 
certainty and worlds with uncertainty. 
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Each of these four parts is divided into chapters. As a quick reference, the table below 
provides an overview of where to find the four solution methods for maximization problems 
used in this book. They are the “substitution method”, “Lagrange approach”, “optimal 
control theory” and “dynamic programming”. Whenever we employ them, we refer to 
them as “Solving by” and then either “substitution”, “the Lagrangian”, “optimal control” 
or “dynamic programming”. As differences and comparative advantages of methods can 
most easily be understood when applied to the same problem, this table also shows the 
most frequently used examples. 

Be aware that these are not the only examples used in this book. Intertemporal profit 
maximization of firms, capital asset pricing, natural volatility, matching models of the 
labour market, optimal R&D expenditure and many other applications can be found as 
well. For a more detailed overview, see the index at the end of this book. 
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Chapter 1 


Introduction 


This book provides a toolbox for solving dynamic maximization problems and for working 
with their solutions in economic models. Maximizing some objective function is central 
to Economics, it can be understood as one of the defining axioms of Economics. When it 
comes to dynamic maximization problems, they can be formulated in discrete or contin- 
uous time, under certainty or uncertainty. Various maximization methods will be used, 
ranging from the substitution method, via the Lagrangian and optimal control to dy- 
namic programming using the Bellman equation. Dynamic programming will be used for 
all environments, discrete, continuous, certain and uncertain, the Lagrangian for most of 
them. The substitution method is also very useful in discrete time setups. The optimal 
control theory, employing the Hamiltonian, is used only for deterministic continuous time 
setups. An overview was given in fig. 0.0.2 on the previous pages. 

The general philosophy behind the style of this book says that what matters is an 
easy and fast derivation of results. This implies that a lot of emphasis will be put on 
examples and applications of methods. While the idea behind the general methods is 
sometimes illustrated, the focus is clearly on providing a solution method and examples 
of applications quickly and easily with as little formal background as possible. This is 
why the book is called applied intertemporal optimization. 


e Contents of parts I to IV 


This book consists of four parts. In this first part of the book, we will get to know 
the simplest and therefore maybe the most useful structures to think about changes over 
time, to think about dynamics. Part I deals with discrete time models under certainty. 
The first chapter introduces the simplest possible intertemporal problem, a two-period 
problem. It is solved in a general way and for many functional forms. The methods used 
are the Lagrangian and simple substitution. Various concepts like the time preference 
rate and the intertemporal elasticities of substitution are introduced here as well, as they 
are widely used in the literature and are used frequently throughout this book. For those 
who want to understand the background of the Lagrangian, a chapter is included that 
shows the link between Lagrangians and solving by substitution. This will also give us the 
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opportunity to explain the concept of shadow prices as they play an important role e.g. 
when using Hamiltonians or dynamic programming. The two-period optimal consumption 
setup will then be put into a decentralized general equilibrium setup. This allows us to 
understand general equilibrium structures in general while, at the same time, we get to 
know the standard overlapping generations (OLG) general equilibrium model. This is one 
of the most widely used dynamic models in Economics. Chapter 2 concludes by reviewing 
some aspects of difference equations. 

Chapter 3 then covers infinite horizon models. We solve a typical maximization prob- 
lem first by using the Lagrangian again and then by dynamic programming. As dynamic 
programming regularly uses the envelope theorem, this theorem is first reviewed in a sim- 
ple static setup. Examples for infinite horizon problems, a general equilibrium analysis 
of a decentralized economy, a typical central planner problem and an analysis of how to 
treat family or population growth in optimization problems then complete this chapter. 
To complete the range of maximization methods used in this book, the presentation of 
these examples will also use the method of “solving by inserting”. 

Part II covers continuous time models under certainty. Chapter 4 first looks at dif- 
ferential equations as they are the basis of the description and solution of maximization 
problems in continuous time. First, some useful definitions and theorems are provided. 
Second, differential equations and differential equation systems are analyzed qualitatively 
by the so-called “phase-diagram analysis”. This simple method is extremely useful for 
understanding differential equations per se and also for later purposes for understand- 
ing qualitative properties of solutions to maximization problems and properties of whole 
economies. Linear differential equations and their economic applications are then finally 
analyzed before some words are spent on linear differential equation systems. 

Chapter 5 presents a new method for solving maximization problems - the Hamil- 
tonian. As we are now in continuous time, two-period models do not exist. A distinction 
will be drawn, however, between finite and infinite horizon models. In practice, this dis- 
tinction is not very important as, as we will see, optimality conditions are very similar for 
finite and infinite maximization problems. After an introductory example on maximiza- 
tion in continuous time by using the Hamiltonian, the simple link between Hamiltonians 
and the Lagrangian is shown. 

The solution to maximization problems in continuous time will consist of one or several 
differential equations. As a unique solution to differential equations requires boundary 
conditions, we will show how boundary conditions are related to the type of maximization 
problem analyzed. The boundary conditions differ significantly between finite and infinite 
horizon models. For the finite horizon models, there are initial or terminal conditions. 
For the infinite horizon models, we will get to know the transversality condition and other 
related conditions like the No-Ponzi-game condition. Many examples and a comparison 
between the present-value and the current-value Hamiltonian conclude this chapter. 

Chapter 6 solves the same kind of problems as chapter 5, but it uses the method 
of “dynamic programming”. The reason for doing this is to simplify understanding of 
dynamic programming in stochastic setups in Part IV. Various aspects specific to the use 


of dynamic programming in continuous time, e.g. the structure of the Bellman equation, 
can already be treated here under certainty. This chapter will also provide a comparison 
between the Hamiltonian and dynamic programming and look at a maximization problem 
with two state variables. An example from monetary economics on real and nominal 
interest rates concludes the chapter. 

In part III, the world becomes stochastic. Parts I and II provided many optimization 
methods for deterministic setups, both in discrete and continuous time. All economic 
questions that were analyzed were viewed as “sufficiently deterministic”. If there was 
any uncertainty in the setup of the problem, we simply ignored it or argued that it is of 
no importance for understanding the basic properties and relationships of the economic 
question. This is a good approach to many economic questions. 

Generally speaking, however, real life has few certain components. Death is certain, 
but when? Taxes are certain, but how high are they? We know that we all exist - but 
don’t ask philosophers. Part III (and part IV later) will take uncertainty in life seriously 
and incorporate it explicitly in the analysis of economic problems. We follow the same 
distinction as in part I and II - we first analyse the effects of uncertainty on economic 
behaviour in discrete time setups in part III and then move to continuous time setups in 
part IV. 

Chapter 7 and 8 are an extended version of chapter 2. As we are in a stochastic world, 
however, chapter 7 will first spend some time reviewing some basics of random variables, 
their moments and distributions. Chapter 7 also looks at difference equations. As they 
are now stochastic, they allow us to understand how distributions change over time and 
how a distribution converges - in the example we look at - to a limiting distribution. 
The limiting distribution is the stochastic equivalent to a fix point or steady state in 
deterministic setups. 

Chapter 8 looks at maximization problems in this stochastic framework and focuses on 
the simplest case of two-period models. A general equilibrium analysis with an overlapping 
generations setup will allow us to look at the new aspects introduced by uncertainty 
for an intertemporal consumption and saving problem. We will also see how one can 
easily understand dynamic behaviour of various variables and derive properties of long- 
run distributions in general equilibrium by graphical analysis. One can for example easily 
obtain the range of the long-run distribution for capital, output and consumption. This 
increases intuitive understanding of the processes at hand tremendously and helps a lot as 
a guide to numerical analysis. Further examples include borrowing and lending between 
risk-averse and risk-neutral households, the pricing of assets in a stochastic world and a 
first look at ‘natural volatility’, a view of business cycles which stresses the link between 
jointly endogenously determined short-run fluctuations and long-run growth. 

Chapter 9 is then similar to chapter 3 and looks at multi-period, i.e. infinite horizon, 
problems. As in each chapter, we start with the classic intertemporal utility maximization 
problem. We then move on to various important applications. The first is a central planner 
stochastic growth model, the second is capital asset pricing in general equilibrium and how 
it relates to utility maximization. We continue with endogenous labour supply and the 
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matching model of unemployment. The next section then covers how many maximization 
problems can be solved without using dynamic programming or the Lagrangian. In fact, 
many problems can be solved simply by inserting, despite uncertainty. This will be 
illustrated with many further applications. A final section on finite horizons concludes. 

Part IV is the final part of this book and, logically, analyzes continuous time models 
under uncertainty. The choice between working in discrete or continuous time is partly 
driven by previous choices: If the literature is mainly in discrete time, students will find 
it helpful to work in discrete time as well. The use of discrete time methods seem to hold 
for macroeconomics, at least when it comes to the analysis of business cycles. On the 
other hand, when we talk about economic growth, labour market analyses and finance, 
continuous time methods are very prominent. 

Whatever the tradition in the literature, continuous time models have the huge ad- 
vantage that they are analytically generally more tractable, once some initial investment 
into new methods has been digested. As an example, some papers in the literature have 
shown that continuous time models with uncertainty can be analyzed in simple phase 
diagrams as in deterministic continuous time setups. See ch. 10.6 and ch. 11.6 on further 
reading for references from many fields. 

To facilitate access to the magical world of continuous time uncertainty, part IV 
presents the tools required to work with uncertainty in continuous time models. It is 
probably the most innovative part of this book as many results from recent research flow 
directly into it. This part also most strongly incorporates the central philosophy behind 
writing this book: There will be hardly any discussion of formal mathematical aspects like 
probability spaces, measurability and the like. While some will argue that one can not 
work with continuous time uncertainty without having studied mathematics, this chapter 
and the many applications in the literature prove the opposite. The objective here is to 
clearly make the tools for continuous time uncertainty available in a language that is ac- 
cessible for anyone with an interest in these tools and some “feeling” for dynamic models 
and random variables. The chapters on further reading will provide links to the more 
mathematical literature. Maybe this is also a good point for the author of this book to 
thank all the mathematicians who helped him gain access to this magical world. I hope 
they will forgive me for “betraying their secrets” to those who, maybe in their view, were 
not appropriately initiated. 

Chapter 10 provides the background for optimization problems. As in part II where 
we first looked at differential equations before working with Hamiltonians, here we will 
first look at stochastic differential equations. After some basics, the most interesting 
aspect of working with uncertainty in continuous time follows: Ito’s lemma and, more 
generally, change-of-variable formulas for computing differentials will be presented. As 
an application of Ito’s lemma, we will get to know one of the most famous results in 
Economics - the Black-Scholes formula. This chapter also presents methods for how to 
solve stochastic differential equations or how to verify solutions and compute moments of 
random variables described by a stochastic process. 


Chapter 11 then looks once more at maximization problems. We will get to know 


the classic intertemporal utility maximization problem both for Poisson uncertainty and 
for Brownian motion. The chapter also shows the link between Poisson processes and 
matching models of the labour market. This is very useful for working with extensions 
of the simple matching model that allows for savings. Capital asset pricing and natural 
volatility conclude the chapter. 


e From simple to complex setups 


Given that certain maximization problems are solved many times - e.g. utility max- 
imization of a household first under certainty in discrete and continuous time and then 
under uncertainty in discrete and continuous time - and using many methods, the “steps 
how to compute solutions” can be easily understood: First, the discrete deterministic 
two-period approach provides the basic intuition or feeling for a solution. Next, infinite 
horizon problems add one dimension of complexity by “taking away” the simple bound- 
ary condition of finite horizon models. In a third step, uncertainty adds expectations 
operators and so on. By gradually working through increasing steps of sophistication and 
by linking back to simple but structurally identical examples, intuition for the complex 
setups is built up as much as possible. This approach then allows us to finally understand 
the beauty of e.g. Keynes-Ramsey rules in continuous time under Poisson uncertainty or 
Brownian motion. 


e Even more motivation for this book 


Why teach a course based on this book? Is it not boring to go through all these 
methods? In a way, the answer is yes. We all want to understand certain empirical 
regularities or understand potential fundamental relationships and make exciting new 
empirically testable predictions. In doing so, we also all need to understand existing work 
and eventually present our own ideas. It is probably much more boring to be hindered 
in understanding existing work and be almost certainly excluded from presenting our 
own ideas if we always spend a long time trying to understand how certain results were 
derived. How did this author get from equation (1) and (2) to equation (3)? The major 
advantage of economic analysis over other social sciences is its strong foundation in formal 
models. These models allow us to discuss questions much more precisely as expressions 
like “marginal utility”, “time preference rate” or “Keynes-Ramsey rule” reveal a lot of 
information in a very short time. It is therefore extremely useful to first spend some time 
in getting to know these methods and then to try to do what Economics is really about: 
understand the real world. 

But, before we really start, there is also a second reason - at least for some economists 
- to go through all these methods: They contain a certain type of truth. A proof is 
true or false. The derivation of some optimal behaviour is true or false. A prediction of 
general equilibrium behaviour of an economy is truth. Unfortunately, it is only truth in 
an analytical sense, but this is at least some type of truth. Better than none. 
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e The audience for this book 


Before this book came out, it had been tested for at least ten years in many courses. 
There are two typical courses which were based on this book. A third year Bachelor 
course (for ambitious students) can be based on parts I and II, i.e. on maximization and 
applications in discrete and continuous time under certainty. Such a course typically took 
14 lectures of 90 minutes each plus the same number of tutorials. It is also possible to 
present the material also in 14 lectures of 90 minutes each plus only 7 tutorials. Presenting 
the material without tutorials requires a lot of energy from the students to go through the 
problem sets on their own. One can, however, discuss some selected problem sets during 
lectures. 

The other typical course which was based on this book is a first-year PhD course. It 
would review a few chapters of part I and part II (especially the chapters on dynamic 
programming) and cover in full the stochastic material of part III and part IV. It also 
requires fourteen 90 minute sessions and exercise classes help even more, given the more 
complex material. But the same type of arrangements as discussed for the Bachelor course 
did work as well. 

Of course, any other combination is feasible. From my own experience, teaching part 
I and II in a third year Bachelor course allows teaching of part III and IV at the Master 
level. Of course, Master courses can be based on any parts of this book, first-year PhD 
courses can start with part I and II and second-year field courses can use material of part 
III or IV. This all depends on the ambition of the programme, the intention of the lecturer 
and the needs of the students. 

Apart from being used in classroom, many PhD students and advanced Bachelor or 
Master students have used various parts of previous versions of this text for studying on 
their own. Given the detailed step-by-step approach to problems, it turned out that it 
was very useful for understanding the basic structure of, say, a maximization problem. 
Once this basic structure was understood, many extensions to more complex problems 
were obtained - some of which then even found their way into this book. 


Part I 


Deterministic models in discrete 
time 


This book consists of four parts. In this first part of the book, we will get to know 
the simplest and therefore maybe the most useful structures to think about changes over 
time, to think about dynamics. Part I deals with discrete time models under certainty. 
The first chapter introduces the simplest possible intertemporal problem, a two-period 
problem. It is solved in a general way and for many functional forms. The methods used 
are the Lagrangian and simple substitution. Various concepts like the time preference 
rate and the intertemporal elasticities of substitution are introduced here as well, as they 
are widely used in the literature and are used frequently throughout this book. For those 
who want to understand the background of the Lagrangian, a chapter is included that 
shows the link between Lagrangians and solving by substitution. This will also give us the 
opportunity to explain the concept of shadow prices as they play an important role e.g. 
when using Hamiltonians or dynamic programming. The two-period optimal consumption 
setup will then be put into a decentralized general equilibrium setup. This allows us to 
understand general equilibrium structures in general while, at the same time, we get to 
know the standard overlapping generations (OLG) general equilibrium model. This is one 
of the most widely used dynamic models in Economics. Chapter 2 concludes by reviewing 
some aspects of difference equations. 

Chapter 3 then covers infinite horizon models. We solve a typical maximization prob- 
lem first by using the Lagrangian again and then by dynamic programming. As dynamic 
programming regularly uses the envelope theorem, this theorem is first reviewed in a sim- 
ple static setup. Examples for infinite horizon problems, a general equilibrium analysis 
of a decentralized economy, a typical central planner problem and an analysis of how to 
treat family or population growth in optimization problems then complete this chapter. 
To complete the range of maximization methods used in this book, the presentation of 
these examples will also use the method of “solving by inserting”. 
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Chapter 2 


Two-period models and difference 
equations 


Given that the idea of this book is to start from simple structures and extend them to 
the more complex ones, this chapter starts with the simplest intertemporal problem, a 
two-period decision framework. This simple setup already allows us to illustrate the basic 
dynamic trade-offs. Aggregating over individual behaviour, assuming an overlapping- 
generations (OLG) structure, and putting individuals in general equilibrium provides an 
understanding of the issues involved in these steps and in identical steps in more general 
settings. Some revision of properties of difference equations concludes this chapter. 


2.1 Intertemporal utility maximization 


2.1.1 The setup 


Let there be an individual living for two periods, in the first she is young, in the second 
she is old. Let her utility function be given by 


U, =U (ese) = U (Ga) s (2.1.1) 


where consumption when young and old are denoted by c/ and c?,, or c; and C41, respec- 
tively, when no ambiguity arises. The individual earns labour income w; in both periods. 
It could also be assumed that she earns labour income only in the first period (e.g. when 
retiring in the second) or only in the second period (e.g. when going to school in the first). 
Here, with s; denoting savings, her budget constraint in the first period is 


Ce + St = Ut (2.1.2) 


and in the second it reads 
Ct+1 = Wt+1 + (1 + T1) St. (2.1.3) 
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Interest paid on savings in the second period are given by 7;,,. All quantities are expressed 
in units of the consumption good (i.e. the price of the consumption good is set equal to 
one. See ch. 2.2.2 for an extension where the price of the consumption good is p;.). 

This budget constraint says something about the assumptions made on the timing of 
wage payments and savings. What an individual can spend in period two is principal and 
interest on her savings s+ of the first period. There are no interest payments in period one. 
This means that wages are paid and consumption takes place at the end of period 1 and 
savings are used for some productive purposes (e.g. firms use it in the form of capital for 
production) in period 2. Therefore, returns r;,; are determined by economic conditions 
in period 2 and have the index t+ 1. Timing is illustrated in the following figure. 


We Ct Wt+1 Ct+1 
St = Wi — Ct (1 + Te41) St 


t t+1 


Figure 2.1.1 Timing in two-period models 


These two budget constraints can be merged into one intertemporal budget constraint 
by substituting out savings, 


We + (1 + Taa)! Wt+1 = Ct + (1 + Tia)! Ct+1- (2.1.4) 


It should be noted that by not restricting savings to be non-negative in (2.1.2) or by equat- 
ing the present value of income on the left-hand side with the present value of consumption 
on the right-hand side in (2.1.4), we assume perfect capital markets: individuals can save 
and borrow any amount they desire. Equation (2.2.14) provides a condition under which 
individuals save. 

Adding the behavioural assumption that individuals maximize utility, the economic 
behaviour of an individual is described completely and one can derive her consumption 
and saving decisions. The problem can be solved by a Lagrange approach or simply by 
substitution. The latter will be done in ch. 2.2.1 and 3.8.2 in deterministic setups or 
extensively in ch. 8.1.4 for a stochastic framework. Substitution transforms an optimiza- 
tion problem with a constraint into an unconstrained problem. We will use a Lagrange 
approach now. 

The maximization problem reads maX«, cı (2.1.1) subject to the intertemporal budget 
constraint (2.1.4). The household’s control variables are c; and c41. As they need to be 
chosen so that they satisfy the budget constraint, they can not be chosen independently 
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of each other. Wages and interest rates are exogenously given to the household. When 
choosing consumption levels, the reaction of these quantities to the decision of our house- 
hold is assumed to be zero - simply because the household is too small to have an effect 
on economy-wide variables. 


2.1.2 Solving by the Lagrangian 


We will solve this maximization problem by using the Lagrange function. This function 
will now be presented simply and its structure will be explained in a “recipe sense” , which 
is the most useful one for those interested in quick applications. For those interested in 
more background, ch. 2.3 will show the formal principles behind the Lagrangian. The 
Lagrangian for our problem reads 


L=U (Ct, Cr41) + A [ew + (1 + Tegi) Wt+1 — Ce — (1 + Ta). C141 5 (2.1.5) 


where A is a parameter called the Lagrange multiplier. The Lagrangian always consists of 
two parts. The first part is the objective function, the second part is the product of the 
Lagrange multiplier and the constraint, expressed as the difference between the right-hand 
side and the left-hand side of (2.1.4). Technically speaking, it makes no difference whether 
one subtracts the left-hand side from the right-hand side as here or vice versa - right- 
hand side minus left-hand side. Reversing the difference would simply change the sign 
of the Lagrange multiplier but not change the final optimality conditions. Economically, 
however, one would usually want a positive sign of the multiplier, as we will see in ch. 2.3. 
The first-order conditions are 


Le = Us, (Ct, Cr41) — À — 0, 
Levy = Vergy (Ct, Ce41) — A [1 + real = 0, 


=wt+(14+ Test) Wet —&— (1+ rei) Cri = 0. 


D` 
> 
| 


Clearly, the last first-order condition is identical to the budget constraint. Note that there 
are three variables to be determined, consumption for both periods and the Lagrange 
multiplier A. Having at least three conditions is a necessary, though not sufficient (they 
might, generally speaking, be linearly dependent) condition for this to be possible. 

The first two first-order conditions can be combined to give 


Ug (ci, Cr41) = (1 + T1141) Üci (ci, Cr41) = As (2.1.6) 


Marginal utility from consumption today on the left-hand side must equal marginal utility 
tomorrow, corrected by the interest rate, on the right-hand side. The economic meaning 
of this correction can best be understood when looking at a version with nominal budget 
constraints (see ch. 2.2.2). 

We learn from this maximization that the modified first-order condition (2.1.6) gives us 
a necessary equation that needs to hold when behaving optimally. It links consumption c 
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today to consumption c;,; tomorrow. This equation together with the budget constraint 
(2.1.4) provides a two-dimensional system: two equations in two unknowns, c; and c41. 
These equations therefore allow us in principle to compute these endogenous variables as 
a function of exogenously given wages and interest rates. This would then be the solution 
to our maximization problem. The next section provides an example where this is indeed 
the case. 


2.2 Examples 


2.2.1 Optimal consumption 
e The setup 


This first example allows us to solve explicitly for consumption levels in both periods. 
Let preferences of households be represented by 


U,=ylng + (1 — y) In C1. (2.2.1) 


This utility function is often referred to as Cobb-Douglas or logarithmic utility function. 
Utility from consumption in each period, instantaneous utility, is given by the logarithm 
of consumption. Instantaneous utility is sometimes also referred to as felicity function. 
As lnc has a positive first and negative second derivative, higher consumption increases 
instantaneous utility but at a decreasing rate. Marginal utility from consumption is 
decreasing in (2.2.1). The parameter y captures the importance of instantaneous utility 
in the first relative to instantaneous utility in the second. Overall utility U; is maximized 
subject to the constraint we know from (2.1.4) above, 


W: = GQ + (1 + raa) Ct+1; (2.2.2) 
where we denote the present value of labour income by 
W, = W + (1 + Tegi) Wt+1- (2.2.3) 


Again, the consumption good is chosen as numéraire good and its price equals unity. 
Wages are therefore expressed in units of the consumption good. 


e Solving by the Lagrangian 
The Lagrangian for this problem reads 
L= qylnc + (1-— y) mei +A (W; —&— (1+ raa)” Ciy] : 
The first-order conditions are 
Le, = (4) -A=0, 
Lo = (1-7) (G1) —AlL + rena] = 0, 
and the budget constraint (2.2.2) following from £, = 0. 
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e The solution 


Dividing first-order conditions gives ae = ] + 74, and therefore 


C= Za (1+ rea) Cea: 
Easy 
This equation corresponds to our optimality rule (2.1.6) derived above for the more general 
case. Inserting into the budget constraint (2.2.2) yields 


= 1 = 
oa ( eae i) (1+ regi) Coy. = —— (14 resi) Cea 
y bay 
which is equivalent to 
Ce = (L—y) ra) We (2.2.4) 


It follows that 
Ct = yW. (2.2.5) 


Apparently, optimal consumption decisions imply that consumption when young is 
given by a share y of the present value W, of life-time income at time t of the individual 
under consideration. Consumption when old is given by the remaining share 1 — y plus 
interest payments, C1 = (1+ r1) (1—7) Wi. Equations (2.2.4) and (2.2.5) are the 
solution to our maximization problem. These expressions are sometimes called “closed- 
form” solutions. A (closed-form) solution expresses the endogenous variable, consumption 
in our case, as a function only of exogenous variables. Closed-form solution is a different 
word for closed-loop solution. For further discussion see ch. 5.6.2. 

Assuming preferences as in (2.2.1) makes modelling sometimes easier than with more 
complex utility functions. A drawback here is that the share of lifetime income consumed 
in the first period and therefore the savings decision is independent of the interest rate, 
which appears implausible. A way out is given by the CES utility function (see below at 
(2.2.10)) which also allows for closed-form solutions for consumption (for an example in a 
stochastic setup, see exercise 6 in ch. 8.1.4). More generally speaking, such a simplification 
should be justified if some aspect of an economy that is fairly independent of savings is 
the focus of some analysis. 


e Solving by substitution 


Let us now consider this example and see how this maximization problem could have 
been solved without using the Lagrangian. The principle is simply to transform an op- 
timization problem with constraints into an optimization problem without constraints. 
This is most simply done in our example by replacing the consumption levels in the ob- 
jective function (2.2.1) by the consumption levels from the constraints (2.1.2) and (2.1.3). 
The unconstrained maximization problem then reads max U; by choosing s+, where 


U; = gln (we — st) + (1 — q) In (wi + (1 + ria) st) - 
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This objective function shows the trade-off faced by anyone who wants to save nicely. 
High savings reduce consumption today but increase consumption tomorrow. 

The first-order condition for savings yields after some simple algebra (see ex. 2 and 
the solutions manual for details) 


St = Wz — Yi, 


where W; is life-time income as defined after (2.2.2). To see that this is consistent with the 
solution by Lagrangian, compute first-period consumption and find c = w — sı = yW; - 
which is the solution in (2.2.5). 

What have we learned from using this substitution method? We see that we do not 
need “sophisticated” tools like the Lagrangian as we can solve a “normal” unconstrained 
problem - the type of problem we might be more familiar with from static maximization 
setups. But the steps to obtain the final solution appear somewhat more “curvy” and less 
elegant. It therefore appears worthwhile to become more familiar with the Lagrangian. 


2.2.2 Optimal consumption with prices 


Consider now again the utility function (2.1.1) and maximize it subject to the constraints 
Dice + St = We and Period = Wer + (1 +7441) s;. The difference to the introductory 
example in ch. 2.1 consists in the introduction of an explicit price p; for the consumption 
good. The first-period budget constraint now equates nominal expenditure for consump- 
tion (pc; is measured in, say, Euro, Dollar or Yen) plus nominal savings to nominal wage 
income. The second period constraint equally equates nominal quantities. What does an 
optimal consumption rule as in (2.1.6) now look like? 
The Lagrangian is, now using an intertemporal budget constraint with prices, 


L = U (ct, C41) +A (W; — pmc — (1+ Titi) Pitic] . 
The first-order conditions for c, and c1 are 


Le, = Us, (cr, Ct41) = Apt = 0, 


Lena = Ce (cr, C41) —A (1 F Pai) Deri =0 


and the intertemporal budget constraint. Combining them gives 


Uc, (Ct, Ce4.1) = Uses (Ct, Ct+1) Ue, (Ces C41) _ Pt 


L = = (2.2.6) 
Pt Per [L A rial : Dees (Cts Ce4t) Pepa [L + rea] 


T: 


This equation says that marginal utility of consumption today relative to marginal utility 
of consumption tomorrow equals the relative price of consumption today and tomorrow. 
This optimality rule is identical for a static 2-good decision problem where optimality 
requires that the ratio of marginal utilities equals the relative price. The relative price 
here is expressed in a present value sense as we compare marginal utilities at two points 
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in time. The price p; is therefore divided by the price tomorrow, discounted by the next 
period’s interest rate, pry |L + real : 

In contrast to what sometimes seems common practice, we will not call (2.2.6) a 
solution to the maximization problem. This expression (frequently referred to as the 
Euler equation) is simply an expression resulting from first-order conditions. Strictly 
speaking, (2.2.6) is only a necessary condition for optimal behaviour - and not more. As 
defined above, a solution to a maximization problem is a closed-form expression as for 
example in (2.2.4) and (2.2.5). It gives information on levels - and not just on changes 
as in (2.2.6). Being aware of this important difference, in what follows, the term “solving 
a maximization problem” will nevertheless cover both analyses. Those which stop at the 
Euler equation and those which go all the way towards obtaining a closed-form solution. 


2.2.3 Some useful definitions with applications 


In order to be able to discuss results in subsequent sections easily, we review some defini- 
tions here that will be used frequently in later parts of this book. We are mainly interested 
in the intertemporal elasticity of substitution and the time preference rate. While a lot 
of this material can be found in micro textbooks, the notation used in these books differs 
of course from the one used here. As this book is also intended to be as self-contained as 
possible, this short review can serve as a reference for subsequent explanations. We start 
with the 


e Marginal rate of substitution (MRS) 


Let there be a consumption bundle (c1, cg, ...., Cn) . Let utility be given by u (c1, C2, ...., Cn) 
which we abbreviate to u (.). The MRS between good i and good j is then defined by 


Ou (.) /Oc; 
Ou (.) /Oc; 


It gives the increase of consumption of good j that is required to keep the utility level 
at u (C1, C2,.---;€n) when the amount of i is decreased marginally. By this definition, this 
amount is positive if both goods are normal goods - i.e. if both partial derivatives in 
(2.2.7) are positive. Note that definitions used in the literature can differ from this one. 
Some replace ’ decreased’ by increased’ (or - which has the same effect - replace ’increase’ 
by ’decrease’) and thereby obtain a different sign. 

Why this is so can easily be shown: Consider the total differential of u (c1, co, ...., Cn) , 
keeping all consumption levels apart from c; and c; fix. This yields 


Ou (. ; 
a Ul) 410, 
Oc; 0c; 
The overall utility level u (c1, C2, ....,n) does not change if 


ae dc; _ ee 


du (c1, C2, ----) Cn) 


dcj 


dci Oc; RSy (.) 
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e Equivalent terms 


As a reminder, the equivalent term to the MRS in production theory is the mar- 
ginal rate of technical substitution M RT Sy; (.) = ae where the utility function was 
replaced by a production function and consumption cą was replaced by factor inputs x,. 

On a more economy-wide level, there is the marginal rate of transformation M RT;; (.) = 
33/80; 
(maybe better known as production possibility curve) and the yx are output of good k. 
The marginal rate of transformation gives the increase in output of good 7 when output 


of good 7 is marginally decreased. 


where the utility function has now been replaced by a transformation function G 


e (Intertemporal) elasticity of substitution 


Though our main interest is a measure of intertemporal substitutability, we first define 
the elasticity of substitution in general. As with the marginal rate of substitution, the 
definition implies a certain sign of the elasticity. In order to obtain a positive sign (with 
normal goods), we define the elasticity of substitution as the increase in relative consump- 


tion c;/c; when the relative price p;/p; decreases (which is equivalent to an increase of 
d(ci/cj) pj/pi 
d(pj/pi) ci/cj ` 
This definition can be expressed alternatively (see ex. 6 for details) in a way which 


is more useful for the examples below. We express the elasticity of substitution by the 
derivative of the log of relative consumption c;/c; with respect to the log of the marginal 
rate of substitution between j and 2, 


p;/pi). Formally, we obtain for the case of two consumption goods €;; = 


bs dln (¢;/c;) 
z dln MRS;j; 


(2.2.8) 


Inserting the marginal rate of substitution M RS}; from (2.2.7), i.e. exchanging 7 and j 
in (2.2.7), gives 
dln (ci/c;) Uc; /Uc, d(ci/c;) 
ile (usli) -alad (ey, [tte,) 
The advantage of an elasticity when compared to a normal derivative, such as the MRS, is 
that an elasticity is measureless. It is expressed in percentage changes. (This can be best 


seen in the following example and in ex. 6 where the derivative is multiplied by Piei ) It 
can both be applied to static utility or production functions or to intertemporal utility 
functions. 

The intertemporal elasticity of substitution for a utility function u (c1, C41) is then 


simply the elasticity of substitution of consumption at two points in time, 


Uc, | Uerss d (C41 / Ce) 
Cri /Ct d (tes / thers) ; 


Et t+1 = (2.2.9) 


Here as well, in order to obtain a positive sign, the subscripts in the denominator have a 
different ordering from the one in the numerator. 
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e The intertemporal elasticity of substitution for logarithmic and CES utility functions 


For the logarithmic utility function U; = yln œ+ (1 — y) Inc41 from (2.2.1), we obtain 
an intertemporal elasticity of substitution of one, 


J= 
ea d (C41/Ct) =i 


salaa] 


Ett+1 = 


where the last step used 


d (Ce41/Ct) = 1— yd (ci41/c) _1- a 


d (2/2) y d(C41/c) y 


Ct! Ct+1 


It is probably worth noting at this point that not all textbooks would agree on the 
result of “plus one”. Following some other definitions, a result of minus one would be 
obtained. Keeping in mind that the sign is just a convention, depending on “increase” or 
“decrease” in the definition, this should not lead to confusions. 

When we consider a utility function where instantaneous utility is not logarithmic but 
of CES type 

U,= yee? + (1) eee, (2.2.10) 


the intertemporal elasticity of substitution becomes 


ad Cr41/C d{y[l=ol gq? /i=7) 12) 44) 
Defining x = (ci41/cr)” , we obtain 
d (Cr41/ Cr) ar 1—y d(ct41/ce) z i= ydre 
d(y[l-o]e?/(l1—-y)(l-o) e41) y adla"/an) Y de 
1— y 1 Aii 
= Eenh, 
y o 


Inserting this into (2.2.11) and cancelling terms, the elasticity of substitution turns out 


to be is 
te feal (=) e 1 


Ett+1 = = 
Geen Gg O Ne G o 


This is where the CES utility function (2.2.10) has its name from: The intertemporal 
elasticity (E) of substitution (S) is constant (C). 
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e The time preference rate 


Intuitively, the time preference rate is the rate at which future instantaneous utilities 
are discounted. To illustrate, imagine a discounted income stream 


Xo : £i t : on eee 
l+r l+r 


where discounting takes place at the interest rate r. Replacing income 2; by instantaneous 
utility and the interest rate by p, p would be the time preference rate. Formally, the 
time preference rate is the marginal rate of substitution of instantaneous utilities (not of 
consumption levels) minus one, 


TPR= M RStt+1 —1. 


As an example, consider the following standard utility function which we will use very 
often in later chapters, 


Uo = D8 obu (a), B= T p>0. (2.2.12) 
Let p be a positive parameter and the implied discount factor, capturing the idea of 
impatience: By multiplying instantaneous utility functions u (c+) by 6", future utility is 
valued less than present utility. This utility function generalizes (2.2.1) in two ways: First 
and most importantly, there is a much longer planning horizon than just two periods. In 
fact, the individual’s overall utility Up stems from the sum of discounted instantaneous 
utility levels u (c,) over periods 0, 1, 2, ... up to infinity. The idea behind this objective 
function is not that individuals live forever but that individuals care about the well- 
being of subsequent generations. Second, the instantaneous utility function u (c+) is not 
logarithmic as in (2.2.1) but of a more general nature where one would usually assume 
positive first and negative second derivatives, u’ > 0, u” < 0. 

The marginal rate of substitution is then 


ikea = U0 /0ul@) __C/ G+)" 1, 


© Uo (.) /du (c) — (1/ (1 +p) 
The time preference rate is therefore given by p. 
Now take for example the utility function (2.2.1). Computing the MRS minus one, we 
have 


pe eee (2.2.13) 
a ome 
The time preference rate is positive if y > 0.5. This makes sense for (2.2.1) as one should 
expect that future utility is valued less than present utility. 

As a side note, all intertemporal utility functions in this book will use exponential 
discounting as in (2.2.12). This is clearly a special case. Models with non-exponential 
or hyperbolic discounting imply fundamentally different dynamic behaviour and time 
inconsistencies. See “further reading” for some references. 
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e Does consumption increase over time? 


This definition of the time preference rate allows us to provide a precise answer to 
the question whether consumption increases over time. We simply compute the condition 
under which c41 > c by using (2.2.4) and (2.2.5), 


Ct+1 > aS (1 = 7) (1 +7441) W; > yW; > 


— 1 
(ee eee ee E TEE a a 
hwy Ly 


= Tity > P- 


Consumption increases if the interest rate is higher than the time preference rate. The 
time preference rate of the individual (being represented by y) determines how to split 
the present value W; of total income into current and future use. If the interest rate is 
sufficiently high to overcompensate impatience, i.e. if (1 — y) (1 + r) > y in the first line, 
consumption rises. 

Note that even though we computed the condition for rising consumption for our 
special utility function (2.2.1), the result that consumption increases when the interest 
rate exceeds the time preference rate holds for more general utility functions as well. We 
will get to know various examples for this in subsequent chapters. 


e Under what conditions are savings positive? 
Savings are from the budget constraint (2.1.2) and the optimal consumption result 


(2.2.5) given by 


+ ope] =” b-7- TI 
St = We — Ce = Wt — wW. — w = w BEEN E 
t t t TERA t I4 Taa t+1 pi I4 Taa 


where the last equality assumed an invariant wage level, w; = Wii, = w. Savings are 
positive if and only if 


Y Y 
>O0<e<1-7> el4 > — S 
St 7 en el Tt+1 1-7 
2y— 1 
Tt > T © Ttip >P (2.2.14) 


This means that savings are positive if interest rate is larger than time preference rate. 
Clearly, this result does not necessarily hold for wy41 > w. 


2.3 The idea behind the Lagrangian 


So far, we simply used the Lagrange function without asking where it comes from. This 
chapter will offer a derivation of the Lagrange function and also an economic interpretation 
of the Lagrange multiplier. In maximization problems employing a utility function, the 
Lagrange multiplier can be understood as a price measured in utility units. It is often 
called a shadow price. 
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2.3.1 Where the Lagrangian comes from I 


e The maximization problem 


Let us consider a maximization problem with some objective function and a constraint, 


max F (x1, £2) subject to g(x1,%2) = b. (2.3.1) 


©1,%2 


The constraint can be looked at as an implicit function, i.e. describing £ as a function 
of xı, ie. 22 = h(x,). Using the representation £2 = h (xı) of the constraint, the 
maximization problem can be written as 


max F (zı,h (x1)). (2.3.2) 


z1 


e The derivatives of implicit functions 


As we will use implicit functions and their derivatives here and in later chapters, we 
briefly illustrate the underlying idea and show how to compute their derivatives. Consider 
a function f (£1, %2,...,%) = 0. The implicit function theorem says - stated simply - 
that this function f (z1, £2, ..., £n) = 0 implicitly defines (under suitable assumptions 
concerning the properties of f (.) - see exercise 7) a functional relationship of the type 
LQ = h (£1, £3, 04,...,0n). We often work with these implicit functions in Economics and 
we are also often interested in the derivative of x2 with respect to, say, £1. 

In order to obtain an expression for this derivative, consider the total differential of 
f (£1, £2, ..-, En) = 0, 


When we keep x3 to £n constant, we can solve this to get 


dx = LOFT) [əx 
a OF) Ons om 


We have thereby obtained an expression for the derivative dr2/dx, without knowing the 
functional form of the implicit function h (£1, £3, %4,..., En) - 


For illustration purposes, consider the following figure. 
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Figure 2.3.1 The implicit function visible at z = 0 


The horizontal axes plot xı and, to the back, x2. The vertical axis plots z. The increas- 
ing surface depicts the graph of the function z = g (£1, £2) — b. When this surface crosses 
the horizontal plane at z = 0, a curve is created which contains all the points where 
z = 0. Looking at this curve illustrates that the function z = 0 © g (#1, £2) = b implicitly 
defines a function £2 = h (x1). See exercise 7 for an explicit analytical derivation of such 
an implicit function. The derivative dx2/dxı is then simply the slope of this curve. The 
analytical expression for this is - using (2.3.3) - dx2/dx, = — (Og (.) /0x1) / (0g (.) /Ox2) . 


e First-order conditions of the maximization problem 


The maximization problem we obtained in (2.3.2) is an example for the substitution 
method: The budget constraint was solved for one control variable and inserted into the 
objective function. The resulting maximization problem is one without constraint. The 
problem in (2.3.2) now has a standard first-order condition, 


dF _ OF | OF dh _ 
dx, > Ox, Ox» dx E 


Taking into consideration that from the implicit function theorem applied to the con- 
straint, 


(2.3.4) 


dh drs Og(#1, £2)/Ox1 
= = — 2.3. 
dx, dx, Og(x1, %2)/Ox’ ee) 


OF OF /Ox2 Og(a1,22) __ 
Da; bgla, oz, — 0. Now define 


the optimality condition (2.3.4) can be written as 


the Lagrange multiplier \ = a a and obtain 
OF Og(x1, £2) 


=A 


=0. 2.3. 
Ox, Ox, ( a0) 
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As can be easily seen, this is the first-order condition of the Lagrangian 
L = F (41,22) + à [b — g (21,22) (2.3.7) 


with respect to 71. 
Now imagine we want to undertake the same steps for x2, i.e. we start from the original 
problem (2.3.1) but substitute out xı. We would then obtain an unconstrained problem 
as in (2.3.2) only that we maximize with respect to x2. Continuing as we just did for xı 

would yield the second first-order condition 
OF Og(x1, £2) 


Ox =e Ox» 


= 0. 


We have thereby shown where the Lagrangian comes from: Whether one defines a La- 
grangian as in (2.3.7) and computes the first-order condition or one computes the first- 
order condition from the unconstrained problem as in (2.3.4) and then uses the implicit 
function theorem and defines a Lagrange multiplier, one always ends up at (2.3.6). The 
Lagrangian-route is obviously faster. 


2.3.2 Shadow prices 
e The idea 


We can now also give an interpretation of the meaning of the multipliers À. Starting 
from the definition of in (2.3.6), we can rewrite it according to 
OF/Ox2 OF _ OF 
Og/Or2 Og = Ob 


A 


One can understand that the first equality can “cancel” the term Ox2 by looking at the 
Of (x1,...,£n) = limAz;—0 f(1,...,0,+Ax,,...,0n)—f(a1,...,0n) Th 
Ox; Fa limAs;>0 Aas ; e 


second equality uses the equality of g and b from the constraint in (2.3.1). From these 
transformations, we see that À equals the change in F as a function of b. It is now easy to 
come up with examples for F or b: How much does F increase (e.g. your utility) when your 
constraint b (your bank account) is relaxed? How much does the social welfare function 
change when the economy has more capital? How much do profits of firms change when 
the firm has more workers? This A is called shadow price and expresses the value of b in 
units of F. 


definition of a (partial) derivative: 
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e A derivation 
A more rigorous derivation is as follows (cf. Intriligator, 1971, ch. 3.3). Compute the 
derivative of the maximized Lagrangian with respect to b, 


OL(x7 (b) 23 (6)) _ ð (F(x (b), £3 (b)) + A(b) [b — g (x2 (b), 73 (b))] 


Ob Ob 
Ox Ory! On; Ox, 


The last equality results from first-order conditions and the fact that the budget constraint 
holds. 

As L(x}, x3) = F (aj, x3) due to the budget constraint holding with equality, 
OL(2}, 13) _ OF (xj, x3) 

Ob g Ob 


(6) = 
e An example 


The Lagrange multiplier A is frequently referred to as shadow price. As we have seen, 
its unit depends on the unit of the objective function F. One can think of price in the 
sense of a price in a currency, for example in Euro, only if the objective function is some 
nominal expression like profits or GDP. Otherwise it is a price expressed for example in 
utility terms. This can explicitly be seen in the following example. Consider a central 
planner that maximizes social welfare u (21,72) subject to technological and resource 
constraints, 

maxu (21, £2) 


subject to 
xı = f (Kı, Lı), z2 = g(K2 Lə), 
Kı+ K= K, Li+ L2 = L. 


Technologies in sectors 1 and 2 are given by f (.) and g(.) and factors of production are 
capital K and labour L. Using as multipliers p1, p2, w£ and w”, the Lagrangian reads 


L = u (z1, z2) + pı [f (Ki L1) — z1] + po [g (Ko, Le) — z2] 


+w” [K — Kı — Kə] + w” [L — Lı — Lo] (2.3.8) 
and first-order conditions are 
eee, Lo m=, (2.3.9) 
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Here we see that the first multiplier pı is not a price expressed in some currency but the 
derivative of the utility function with respect to good 1, i.e. marginal utility. By contrast, 
if we looked at the multiplier w% only in the third first-order condition, w* = p,0f/0K,, 
we would then conclude that it is a price. Then inserting the first first-order condition, 
Ou/Ox, = pı, and using the constraint zı = f (Kı, Lı) shows however that it really stands 
for the increase in utility when the capital stock used in production of good 1 rises, 


Of Ou Of ðu 
Pinna, Op OlG ORG 


w” = 


Hence w* and all other multipliers are prices in utility units. 

It is now also easy to see that all shadow prices are prices expressed in some currency if 
the objective function is not utility but, for example GDP. Such a maximization problem 
could read max pı£ı + p2X2 subject to the constraints as above. Finally, returning to 
the discussion after (2.1.5), the first-order conditions show that the sign of the Lagrange 
multiplier should be positive from an economic perspective. If pı in (2.3.9) is to capture 
the value attached to x, in utility units and x, is a normal good (utility increases in z4, i.e. 
Ou/Ox, > 0), the shadow price should be positive. If we had represented the constraint in 
the Lagrangian (2.3.8) as xı — f (4%,£1) rather than right-hand side minus left-hand side, 
the first-order condition would read Ou/Ox,; + pı = 0 and the Lagrange multiplier would 
have been negative. If we want to associate the Lagrange multiplier to the shadow price, 
the constraints in the Lagrange function should be represented such that the Lagrange 
multiplier is positive. 


2.4 An overlapping generations model 


We will now analyze many households jointly and see how their consumption and saving 
behaviour affects the evolution of the economy as a whole. We will get to know the Euler 
theorem and how it is used to sum factor incomes to yield GDP. We will also understand 
how the interest rate in the household’s budget constraint is related to marginal produc- 
tivity of capital and the depreciation rate. All this jointly yields time paths of aggregate 
consumption, the capital stock and GDP. We will assume an overlapping-generations 
structure (OLG). 

A model in general equilibrium is described by fundamentals of the model and mar- 
ket and behavioural assumptions. Fundamentals are technologies of firms, preferences of 
households and factor endowments. Adding clearing conditions for markets and behav- 
ioural assumptions for agents completes the description of the model. 
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2.4.1 Technologies 
e The firms 


Let there be many firms who employ capital K; and labour L to produce output Y; 
according to the technology 
Y= Y (K, L). (2.4.1) 


Assume production of the final good Y (.) is characterized by constant returns to scale. 
We choose Y, as our numéraire good and normalize its price to unity, p = 1. While this 
is not necessary and we could keep the price p; all through the model, we would see that 
all prices, like for example factor rewards, would be expressed relative to the price p;. 
Hence, as a shortcut, we set p, = 1. We now, however, need to keep in mind that all prices 
are henceforth expressed in units of this final good. With this normalization, profits are 
given by m = Y, — wk K, — wł L. Letting firms act under perfect competition, the first- 
order conditions from profit maximization reflect the fact that the firm takes all prices as 
parametric and set marginal productivities equal to real factor rewards, 


ai (2.4.2) 
K 


In each period they equate t, the marginal productivity of capital, to the factor price w; 
for capital and the marginal productivity of labour to labour’s factor reward wf. 


e Euler’s theorem 


Euler’s theorem shows that for a linear-homogeneous function f (£1, 22, ...,Un) the 
sum of partial derivatives times the variables with respect to which the derivative was 
computed equals the original function f (.) , 


_ OF), , FW 


_ OF () 
Ox, Ox 


Ba 
Ofn 


T (Git wee) Lot... (2.4.3) 
Provided that the technology used by firms to produce Y; has constant returns to scale, 
we obtain from this theorem that 


ROA 


OY; 
Y, = kee 


; .4.4 
oK; OL 2 eae) 


Using the optimality conditions (2.4.2) of the firm for the applied version of Euler’s 
theorem (2.4.4) yields 
Y, = wE K, + wP L. (2.4.5) 


Total output in this economy, Y;, is identical to total factor income. This result is usually 
given the economic interpretation that under perfect competition all revenue in firms is 
used to pay factors of production. As a consequence, profits 7; of firms are zero. 
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2.4.2 Households 


e Individual households 


Households live again for two periods. The utility function is therefore as in (2.2.1) 
and given by 
U, = ylncf + (1— y) nee). (2.4.6) 


It is maximized subject to the intertemporal budget constraint 


we = Y + (1471) ePi 

This constraint differs slightly from (2.2.2) in that people work only in the first period and 

retire in the second. Hence, there is labour income only in the first period on the left-hand 

side. Savings from the first period are used to finance consumption in the second period. 
Given that the present value of lifetime wage income is w/, we can conclude from 

(2.2.4) and (2.2.5) that individual consumption expenditure and savings are given by 


d= yur, Ca =(= tre) wf, (2.4.7) 
s, = wr — cf = (1—7) wt. (2.4.8) 


e Aggregation 


We assume that in each period L individuals are born and die. Hence, the number 
of young and the number of old is L as well. As all individuals within a generation 
are identical, aggregate consumption within one generation is simply the number of, say, 
young times individual consumption. Aggregate consumption in t is therefore given by 
C; = Lc! + Le. Using the expressions for individual consumption from (2.4.7) and noting 
the index t (and not t + 1) for the old yields 


C; = Le} + Le? = (quy + (1—7) (1+ re) wf) L. 


2.4.3 Goods market equilibrium and accumulation identity 


The goods market equilibrium requires that supply equals demand, Y; = C; + I, where 
demand is given by consumption plus gross investment. Next period’s capital stock is - by 
an accounting identity - given by Kı = + (1 — 6) Kı. Net investment, amounting to the 
change in the capital stock, Ay,1 — K, is given by gross investment J; minus depreciation 
ôK, where ô is the depreciation rate, Kı — Ki = lL — ôK.. Replacing gross investment 
by the goods market equilibrium, we obtain the resource constraint 


Kipi = (1 — ô) K; + Yı = Ci. (2.4.9) 
For our OLG setup, it is useful to rewrite this constraint slightly , 
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In this formulation, it reflects a “broader” goods market equilibrium where the left-hand 
side shows supply as current production plus capital held by the old. The old sell capital 
as it is of no use for them, given that they will not be able to consume anything in the 
next period. Demand for the aggregate good is given by aggregate consumption (i.e. 
consumption of the young plus consumption of the old) plus the capital stock to be held 
next period by the currently young. 


2.4.4 The reduced form 


For the first time in this book we have come to the point where we need to find what will be 
called a “reduced form”. Once all maximization problems are solved and all constraints 
and market equilibria are taken into account, the objective consists of understanding 
properties of the model, i.e. understanding its predictions. This is usually done by first 
simplifying the structure of the system of equations coming out of the model as much 
as possible. In the end, after inserting and reinserting, a system of n equations in n 
unknowns results. The system where n is the smallest possible is what will be called the 
reduced form. 

Ideally, there is only one equation left and this equation gives an explicit solution of the 
endogenous variable. In static models, an example would be Lx = aL, i.e. employment 
in sector X is given by a utility parameter a times the total exogenous labour supply L. 
This would be an explicit solution. If we are left with just one equation but we obtain on 
an implicit solution, we would obtain something like f (Lx,a,L) = 0. 


e Deriving the reduced form 


We now derive, given the results we have obtained so far, how large the capital stock 
in the next period is. Splitting aggregate consumption into consumption of the young 
and consumption of the old and using the output-factor reward identity (2.4.5) for the 
resource constraint in the OLG case (2.4.10), we obtain 


we Kh, +wtL + (1 -— ô) Ke = CP + Ce + Kua. 


Defining the interest rate r+ as the difference between factor rewards wE for capital and 
the depreciation rate 6, 

r, = uf — ô, (2.4.11) 
we find 


rK bor Ls K; = C? +C? + Kaa. 


The interest rate definition (2.4.11) shows the net income of capital owners per unit of 
capital. They earn the gross factor rewards w but, at the same time, they experience a 
loss from depreciation. Net income therefore only amounts to r;. As the old consume the 
capital stock plus interest c? L = (1+ r,) Ay, we obtain 


Kaa S40 b= C? = sL. (2.4.12) 
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which is the aggregate version of the savings equation (2.4.8). Hence, we have found that 
savings s+ of young at t is the capital stock at t + 1. 

Note that equation (2.4.12) is often present on “intuitive” grounds. The old in period 
t have no reason to save as they will not be able to use their savings in t + 1. Hence, only 
the young will save and the capital stock in t + 1, being made up from savings in the 
previous period, must be equal to the savings of the young. 


e The one-dimensional difference equation 


In our simple dynamic model considered here, we obtain the ideal case where we are 
left with only one equation that gives us the solution for one variable, the capital stock. 
Inserting the individual savings equation (2.4.8) into (2.4.12) gives with the first-order 
condition (2.4.2) of the firm 


OY (Ka, L) 


Kea =(=) hE = (1-9) 


L (2.4.13) 
The first equality shows that a share 1 — y of labour income turns into capital in the next 
period. Interestingly, the depreciation rate does not have an impact on the capital stock 
in period t+ 1. Economically speaking, the depreciation rate affects the wealth of the old 
but - with logarithmic utility - not the saving of the young. 


2.4.5 Properties of the reduced form 


Equation (2.4.13) is a non-linear difference equation in K;. All other quantities in this 
equation are constant. This equation determines the entire path of capital in this dynamic 
economy, provided we have an initial condition Ko. We have therefore indeed solved the 
maximization problem and reduced the general equilibrium model to one single equation. 
From the path of capital, we can compute all other variables which are of interest for our 
economic questions. 

Whenever we have reduced a model to its reduced form and have obtained one or 
more difference equations (or differential equations in continuous time), we would like to 
understand the properties of such a dynamic system. The procedure is in principle always 
the same: We first ask whether there is some solution where all variables (K, in our case) 
are constant. This is then called a steady state analysis. Once we have understood the 
steady state (if there is one), we want to understand how the economy behaves out of the 
steady state, i.e. what its transitional dynamics are. 


e Steady state 


In the steady state, the capital stock is constant, Kı = Kı = K*, and determined 
by 
OY (K*, L) 


KY = (1-7) —pp 


ie (2.4.14) 
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All other variables like aggregate consumption, interest rates, wages etc. are constant as 
well. Consumption when young and when old can differ, as in a setup with finite lifetimes, 
the interest rate in the steady state does not need to equal the time preference rate of 
households. 


e Transitional dynamics 


Dynamics of the capital stock are illustrated in figure 2.4.1. The figure plots the capital 
stock in period ¢ on the horizontal axis. The capital stock in the next period, K:+1, is 
plotted on the vertical axis. The law of motion for capital from (2.4.13) then shows up as 
the curve in this figure. The 45° line equates Kı to Ky. 

We start from our initial condition Ko. Equation (2.4.13) or the curve in this figure 
then determines the capital stock Kı. This capital stock is then viewed as K, so that, 
again, the curve gives us K;1, which is, given that we now started in 1, the capital stock 
Ko of period 2. We can continue doing so and see graphically that the economy approaches 
the steady state K* which we computed in (2.4.14). 


Kiri 
Ki = Kia 


Kaa = (1 — yw (K) L 


Ko K: 


Figure 2.4.1 Convergence to the steady state 
e Summary 


We started with a description of technologies in (2.4.1), preferences in (2.4.6) and 
factor endowment given by Ko. With behavioural assumptions concerning utility and 
profit maximization and perfect competition on all markets plus a description of markets 
in (2.4.3) and some “juggling of equations”, we ended up with a one-dimensional difference 
equation (2.4.13) which describes the evolution of the economy over time and steady state 
in the long-run. Given this formal analysis of the model, we could now start answering 
economic questions. 
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2.5 More on difference equations 


The reduced form in (2.4.13) of the general equilibrium model turned out to be a non- 
linear difference equation. We derived its properties in a fairly intuitive manner. However, 
one can approach difference equations in a more systematic manner, which we will do in 
this chapter. 


2.5.1 Two useful proofs 


Before we look at difference equations, we provide two results on sums which will be 
useful in what follows. As the proof of this result also has an esthetic value, there will be 
a second proof of another result to be done in the exercises. 


Lemma 2.5.1 For anya #1, 


Phas = Ya = n 
Proof. The left hand side is given by 
Daa Hata te tte Pan (2.5.1) 
Multiplying this sum by a yields 
adja’ =a? +a? +... +a" +a., (2.5.2) 


Now subtract (2.5.2) from (2.5.1) and find 


(l=@)2 a maa 7 en ge (2.5.3) 
E 
Lemma 2.5.2 

Dlia = G = T a 
l1—a l-a 
Proof. The proof is left as exercise 9. m 
2.5.2 A simple difference equation 
One of the simplest difference equations is 
fay =a. a> 0. (2.5.4) 


This equation appears too simple to be worth analysing. We do it here as we get to 
know the standard steps in analyzing difference equations which we will also use for more 
complex difference equations. The objective here is therefore not this difference equation 
as such but what is done with it. 
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e Solving by substitution 


The simplest way to find a solution to (2.5.4) consists of inserting and reinserting this 
equation sufficiently often. Doing it three times gives 


Tı = aXo, 
T2 = ary, = a° xo, 


T3 = AT = a? xo. 
When we look at this solution for t = 3 long enough, we see that the general solution is 
Xi = ato. (2.5.5) 


This could formally be proven by either induction or by verification. In this context, we 
can make use of the following 


Definition 2.5.1 A solution of a difference equation is a function of time which, when 
inserted into the original difference equation, satisfies this difference equation. 


Equation (2.5.5) gives z as a function of time t only. Verifying that it is a solution 
indeed just requires inserting it twice into (2.5.4) to see that it satisfies the original 
difference equation. 


e Examples for solutions 
The sequence of x; given by this solution, given different initial conditions x9, are shown 


in the following figure for a > 1. The parameter values chosen are a = 2, x € {0.5, 1,2} 
and t runs from 0 to 10. 
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Figure 2.5.1 Solutions to a difference equation for a> 1 
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e Long-term behaviour 


We can now ask whether x; approaches a constant when time goes to infinity. This 
gives 


0 O<a<l 
lim z; = zo lma’=< zo > a=1 
t—oo t—00 

(oe) a>l 


Hence, x; approaches a constant only when a < 1. For a = 1, it stays at its initial value 
To. 


e A graphical analysis 


For more complex difference equations, it often turns out to be useful to analyze 
their behaviour in a phase diagram. Even though this simple difference equation can be 
understood easily analytically, we will illustrate its properties in the following figure. Here 
as well, this allows us to understand how analyses of this type can also be undertaken for 
more complex difference equations. 


Tiji Tt+1 


Lt = Lt41 


Xo Xt 
Figure 2.5.2 A phase diagram for a < 1 on the left anda > 1 in the right panel 


The principle of a phase diagram is simple. The horizontal axis plots x, the vertical 
axis plots £41. There is a 45° line which serves to equate x, to £41 and there is a plot of 
the difference equation we want to understand. In our current example, we plot £41 as 
axı into the figure. Now start with some initial value 79 and plot this on the horizontal 
axis as in the left panel. The value for the next period, i.e. for period 1, can then be read 
off the vertical axis by looking at the graph of ax. This value for x; is then copied onto 
the horizontal axis by using the 45° line. Once on the horizontal axis, we can again use 
the graph of ax; to compute the next x;,,. Continuing to do so, the left panel shows how 
x, evolves over time, starting at xo. In this case of a < 1, we see how x; approaches zero. 
When we graphically illustrate the case of a > 1, the evolution of x; is as shown in the 
right panel. 
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2.5.3 A slightly less but still simple difference equation 


We now consider a slightly more general difference equation. Compared to (2.5.4), we 
just add a constant b in each period, 


T1 =ax,+b, a>O. (2.5.6) 


We can plot phase diagrams for this difference equation for different parameter values. 


Tt+41 = AX + b 


Tt+1 


Lt = M41 


TLt+1 = ATi + b 


Ge Ti x T 


Figure 2.5.3 Phase diagrams of (2.5.6) for positive (left panel) and negative b (right 
panel) and higher a in the right panel 


We will work with them shortly. 
e Solving by substitution 


We solve again by substituting. In contrast to the solution for (2.5.4), we start from 
an initial value of x,. Hence, we imagine we are in t (t as today) and compute what the 
level of x will be tomorrow and the day after tomorrow etc. We find for x42 and 213 
that 


Dive = O24, +b = a [azi +b] +b = a°x, +b[1 +a], 
Tis = at, +b |1 +a +a’]. 


This suggests that the general solution is of the form 


a” — 1 
a—1` 


Ltin = A” + bE dai = ax +b 


The last equality used the first lemma from ch. 2.5.1. 
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e Limit for n — œ anda<1 


The limit for n going to infinity and a < 1 is given by 


lim Zin = lim a", 4 pe = (2.5.7) 
n= n—0o a 
e A graphical analysis 


The left panel in fig. 2.5.3 studies the evolution of x; for the stable case, i.e. where 
0<a< 1and b > 0. Starting today in t with x+, we end up in x*. As we chose a smaller 
than one and a positive b, x* is positive as (2.5.7) shows. We will return to the right panel 
in a moment. 


2.5.4 Fix points and stability 


e Definitions 


We can use these examples to define two concepts that will also be useful at later 
stages. 


Definition 2.5.2 (Fixpoint) A fixpoint x* of a function f (x) is defined by 
Se’). (2.5.8) 


For difference equations of the type tı = f (2), the fixpoint x* of the function 
f (x) is also the point where x stays constant, i.e. £41 = 2. This is usually called the 
long-run equilibrium point of some economy or its steady or stationary state. Whenever 
an economic model, represented by its reduced form, is analyzed, it is generally useful to 
first try and find out whether fixpoints exist and what their economic properties are. For 
the difference equation from the last section, we obtain 


Ti = Tp So oe a =ar +be w= 7 : 
—a 


Once a fixpoint has been identified, one can ask whether it is stable. 


Definition 2.5.3 (Global stability) A fixpoint x* is globally stable if, starting from an 
initial value £o # x*, x; converges to x*. 


The concept of global stability usually refers to initial values x9 that are economically 
meaningful. An initial physical capital stock that is negative would not be considered to 
be economically meaningful. 
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locally stable 
diverges from } z“ 


Definition 2.5.4 (Local stability and instability) A fixpoint x* is l meta \ if, 


starting from an initial value x* + £, where € is small, x; 
bl Pi 
converges to 


For illustration purposes consider the fixpoint x* in the left panel of fig. 2.5.3 - it is 
globally stable. In the right panel of the same figure, it is unstable. As can easily be seen, 
the instability follows by simply letting the x;,, line intersect the 45°-line from below. In 
terms of the underlying difference equation (2.5.6), this requires b < 0 anda > 1. 

Clearly, economic systems can be much more complex and generate several fixpoints. 
Imagine the link between z1 and z+ is not linear as in (2.5.6) but nonlinear, x44, = f (x+). 
Unfortunately for economic analysis, a nonlinear relationship is the much more realistic 
case. The next figure provides an example for some function f (x+) that implies an unstable 
x* and a locally stable fixpoint x%. 


* x x 
XX, Xoz x, 03 t 


Figure 2.5.4 A locally stable fixpoint x% and an unstable fixpoint «* 


2.5.5 An example: Deriving a budget constraint 


A frequently encountered difference equation is the budget constraint. We have worked 
with budget constraints at various points before but we have hardly thought about their 
origin. We more or less simply wrote them down. Budget constraints, however, are tricky 
objects, at least when we think about general equilibrium setups. What is the asset we 
save in? Is there only one asset or are there several? What are the prices of these assets? 
How does it relate to the price of the consumption good, i.e. do we express the value of 
assets in real or nominal terms? 

This section will derive a budget constraint. We assume that there is only one asset. 
The price of one unit of the asset will be denoted by 4. Its relation to the price p 
of the consumption good will be left unspecified, i.e. we will discuss the most general 
setup which is possible for the one-asset case. The derivation of a budget constraint is in 
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principle straightforward. One defines the wealth of the household (taking into account 
which types of assets the household can hold for saving purposes and what their prices 
are), computes the difference between wealth “today” and “tomorrow” (this is where the 
difference equation aspect comes in) and uses an equation which relates current savings 
to current changes in the number of assets. In a final step, one will naturally find out how 
the interest rate appearing in budget constraints relates to more fundamental quantities 
like value marginal products and depreciation rates. 


e A real budget constraint 


The budget constraint which results depends on the measurement of wealth. We start 
with the case where we measure wealth in units of share, or “number of machines” ky. 
Savings of a household who owns k; shares are given by capital income (net of depreciation 
losses) plus labour income minus consumption expenditure, 


eee. L 
St = Wi ki = Ôviki + Wr — PtCt- 


This is an identity resulting from bookkeeping of flows at the household level. Savings in 
t are used for buying new assets in t for which the period-t price v, needs to be paid, 


are ene A (2.5.9) 
Ut 


We can rewrite this equation slightly, which will simplify the interpretation of subsequent 
results, as 


K Tt 
kiri = (1 — ô) ki 4 Gone aS 


Ut 
Wealth in the next period expressed in number of stocks (and hence not in nominal terms) 
is given by wealth which is left over from the current period, (1 — 6) k;, plus new acqui- 
sitions of stocks which amount to gross capital plus labour income minus consumption 
expenditure divided by the price of one stock. Collecting the k; terms and defining an 
interest rate 


gives a budget constraint for wealth measured by ky, 


K L L 
key = (1 fete: 5) i= Pa EE N ee = eee. (2.5.10) 
Ut Ut Ut Ut Ut 


This is a difference equation in k; but not yet a difference equation in nominal wealth a. 
Rearranging such that expenditure is on the left- and disposable income on the right- 
hand side yields 
pice + Vekt = Vekt + (wy — v6) ke + we. 


This equation also lends itself to a simple interpretation: On the left-hand side is total 
expenditure in period t, consisting of consumption expenditure p,c; plus expenditure for 
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buying the number of capital goods, ki11, the household wants to hold in t + 1. As this 
expenditure is made in t, total expenditure for capital goods amounts to v;k;41. The right- 
hand side is total disposable income which splits into income v;k; from selling all capital 
“inherited” from the previous period, capital income (wE — vð) k; and labour income w?. 
This is the form budget constraints are often expressed in capital asset pricing models. 
Note that this is in principle also a difference equation in ky. 


e A nominal budget constraint 


In our one-asset case, nominal wealth a, of a household is given by the number k; of 
stocks the household owns (say the number of machines it owns) times the price v, of one 
stock (or machine), a; = v;k;. Computing the first difference yields 


Qt41 — at = Uti keq1 — veki 
= Vt+1 (kiyi r ky) + (vipi = Ut) ki (2.5.11) 


where the second line added v+1kt — vi11k;:. Wealth changes depend on the acquisition 
Ut41 (ki+1 — kt) of new assets and on changes in the value of assets that are already held, 
(Ui41 — Uz) ky. Inserting (2.5.9) into (2.5.11) yields 


Ut4+1 


At41—-— A = FA + (vipi — ve) ki 
t 
Ut+1 wë L Ut+1 
= — | —u — ða + wy — pe | + | — -1]a se 

Ut Ut Ut 

Vt+1 we L 

Qiy = 14 — 6) ag + wy — pG |. (2.5.12) 

Ut Ut 


What does this equation tell us? Each unit of wealth a; (say Euro, Dollar, Yen ...) gives 1— 
6 units at the end of the period as 6% is lost due to depreciation plus “dividend payments” 
wE /v. Wealth is augmented by labour income minus consumption expenditure. This 
end-of-period wealth is expressed in wealth a;,, at the beginning of the next period by 
dividing it through v; (which gives the number k, of stocks at the end of the period) and 
multiplying it by the price vı of stocks in the next period. We have thereby obtained a 
difference equation in ay. 

This general budget constraint is fairly complex, however, which implies that in prac- 
tice it is often expressed differently. One possibility consists of choosing the capital good 
as the numéraire good and setting v, = 1 Vt. This simplifies (2.5.12) to 


atya = (1 + ri) ae + wE — pect. (2.5.13) 


The simplification in this expression consists also in the definition of the interest rate r; 
ah 
as r; = w — ô. 
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2.6 Further reading and exercises 


Rates of substitution are discussed in many books on Microeconomics; see e.g. Mas-Colell, 
Whinston and Green (1995) or Varian (1992). The definition of the time preference rate is 
not very explicit in the literature. An alternative formulation implying the same definition 
as the one we use here is used by Buiter (1981, p. 773). He defines the pure rate of time 
preference as “the marginal rate of substitution between consumption” in two periods 
“when equal amounts are consumed in both periods, minus one.” A derivation of the 
time preference rate for a two-period model is in appendix A.1 of Bossmann, Kleiber and 
Wälde (2007). 

The OLG model goes back to Samuelson. For presentations in textbooks, see e.g. Blan- 
chard and Fischer (1989), Azariadis (1993) or de la Croix and Michel (2002). Applications 
of OLG models are more than numerous. For an example concerning bequests and wealth 
distributions, see Bossmann, Kleiber and Wälde (2007). See also Galor and Moav (2006) 
and Galor and Zeira (1993). 

The presentation of the Lagrangian is inspired by Intriligator (1971, p. 28 - 30). Treat- 
ments of shadow prices are available in many other textbooks (Dixit, 1989, ch. 4; Intrili- 
gator, 1971, ch. 3.3). More extensive treatments of difference equations and the implicit 
function theorem can be found in many introductory “mathematics for economists” books. 

There is an interesting discussion on the empirical relevance of exponential discount- 
ing. An early analysis of the implications of non-exponential discounting is by Strotz 
(1955/56). An overview is provided by Frederick et al. (2002). An analysis using sto- 
chastic continuous time methods is by Gong et al. (2007). 
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Exercises chapter 2 
Applied Intertemporal Optimization 


Optimal consumption in two-period discrete time models 


1. Optimal choice of household consumption 
Consider the following maximization problem, 


1 
U, = 
max Ut ule) + 77 


Ct,Ct+1 


r (Cr41) (2.6.1) 


subject to 


wet (Ltr) we =o (1 +r) ct. 


Solve it by using the Lagrangian. 


(a) What is the optimal consumption path? 
(b) Under what conditions does consumption rise? 


(c) Show that the first-order conditions can be written as u’ (c4) /u’ (cr41) = 8 [1 +r]. 
What does this equation tell you? 


2. Solving by substitution 
Consider the maximization problem of section 2.2.1 and solve it by substitution. 
Solve the constraint for one of the control variables, insert this into the objective 
function and compute first-order conditions. Show that the same results as in (2.2.4) 
and (2.2.5) are obtained. 


3. Capital market restrictions 
Now consider the following budget constraint. This is a budget constraint that 
would be appropriate if you want to study the education decisions of households. 
The parameter b amounts to schooling costs. Inheritance of this individual under 
consideration is n. 
U,=ylng+ (1 — y) ln C1 


subject to 
—b +n + (1 + ry) Wt+1 = Ct + (1 + ry) Ct41- 
(a) What is the optimal consumption profile under no capital market restrictions? 


(b) Assume loans for financing education are not available, hence savings need to 
be positive, s; > 0. What is the consumption profile in this case? 
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4. Optimal investment 
Consider a monopolist investing in its technology. Technology is captured by mar- 
ginal costs cœ. The chief accountant of the firm has provided the manager of the 
firm with the following information, 


M= rit Rro, Ti = p(t) ti — Gti l, G41=a¢—f(h). 


Assume you are the manager. What is the optimal investment sequence J, I2? 


5. A particular utility function 
Consider the utility function U = c + Ge41, where 0 < 8 < 1. Maximize U subject 
to an arbitrary budget constraint of your choice. Derive consumption in the first 
and second period. What is strange about this utility function? 


6. Intertemporal elasticity of substitution 
Consider the utility function U = 47 + Bc;7Y. 


(a) What is the intertemporal elasticity of substitution? 
(b) How can the definition in (2.2.8) of the elasticity of substitution be transformed 


into the maybe better known definition 


dln (ci/cj) _ pj/pi d(ci/ey) , 


eos i 
1 din (4th) ci/cj d (p;/pi) 


What does ¢€;; stand for in words? 


7. An implicit function 
Consider the constraint £2 — £f — xı = b. 


(a) Convince yourself that this implicitly defines a function rz = h (x1). Can the 
function h (x1) be made explicit? 


(b) Convince yourself that this implicitly defines a function zı = k (x2). Can the 
function k (£2) be made explicit? 


(c) Think of a constraint which does not define an implicit function. 


8. General equilibrium 
Consider the Diamond model for a Cobb-Douglas production function of the form 
Y; = KP L' and a logarithmic utility function u = ln c¥ + 6 Inc?,4. 
(a) Derive the difference equation for K;. 
(b) Draw a phase diagram. 


(c) What are the steady state consumption level and capital stock? 
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9. Sums 


(a) Proof the statement of the second lemma in ch. 2.5.1, 


4 ah eae 
poaa = (o aes a) 
l1—a 


The idea is identical to the first proof in ch. 2.5.1. 
(b) Show that 


k k 
k-1 k-1-s, s _ 4 TY 
MG CA y= : 

C4 — V 


Both parameters obey 0 < c4 < 1 and 0 < v < 1. Hint: Rewrite the sum as 
cy *D*=5 (v/ca)® and observe that the first lemma in ch. 2.5.1 holds for a which 
are larger or smaller than 1. 


10. Difference equations 
Consider the following linear difference equation system 


Yi =ay+b, a<0<8, 


(a) What is the fixpoint of this equation? 
(b) Is this point stable? 


(c) Draw a phase diagram. 
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Chapter 3 


Multi-period models 


This chapter looks at decision processes where the time horizon is longer than two periods. 
In most cases, the planning horizon will be infinity. In such a context, Bellman’s optimality 
principle is very useful. Is it, however, not the only way to solve maximization problems 
with infinite time horizon? For comparison purposes, we therefore start with the Lagrange 
approach, as in the last section. Bellman’s principle will be introduced afterwards when 
intuition for the problem and relationships will have been increased. 


3.1 Intertemporal utility maximization 


3.1.1 The setup 


The objective function is given by the utility function of an individual, 
U= ae Tule; (3.1.1) 


where again as in (2.2.12) 
B= (Go) p>0 (3.1.2) 


is the discount factor and p is the positive time preference rate. We know this utility 
function already from the definition of the time preference rate, see (2.2.12). The util- 
ity function is to be maximized subject to a budget constraint. The difference to the 
formulation in the last section is that consumption does not have to be determined for 
two periods only but for infinitely many. Hence, the individual does not choose one or 
two consumption levels but an entire path of consumption. This path will be denoted by 
{c}. As T >t, {c} is a short form of {c:, ¢41,...}. Note that the utility function is a 
generalization of the one used above in (2.1.1), but is assumed to be additively separable. 
The corresponding two period utility function was used in exercise set 1, cf. equation 
(2.6.1). 
The budget constraint can be expressed in the intertemporal version by 


EL, try %e, = a + EL, (147) 0 w, (3.1.3) 
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where e, = p,c,. It states that the present value of expenditure equals current wealth 
a, plus the present value of labour income w,. Labour income w, and the interest rate 
r are exogenously given to the household, its wealth level a; is given by history. The 
only quantity that is left to be determined is therefore the path {c,}. Maximizing (3.1.1) 
subject to (3.1.3) is a standard Lagrange problem. 


3.1.2 Solving by the Lagrangian 
The Lagrangian reads 
L= D267 ule.) +A pas (Perr Mee aa a | 
where A is the Lagrange multiplier. First-order conditions are 
Lo, = Btu (c) +A [+r] p, =0, t<T< o, (3.1.4) 
Pech, (3.1.5) 


where the latter is, as in the OLG case, the budget constraint. Again, we have as many 
conditions as variables to be determined: there are infinitely many conditions in (3.1.4), 
one for each c, and one condition for A in (3.1.5). 

Do these first-order conditions tell us something? Take the first-order condition for 
period T and for period 7 + 1. They read 


BT tu’ (c) = —A[1 + ane Pr, 
potty (e41) SaN [1 + A an Pr+t 


Dividing them gives 


awhe) _ gy Pr ag Wer) O 
f w (Crea) — Gtr) Pr+1 Z Bu! (cr41) (Ltr) pry (3-1.6) 


Rearranging allows us to see an intuitive interpretation: Comparing the instantaneous 
gain in utility u’ (c-) with the future gain, discounted at the time preference rate, Bu’ (c41), 
must yield the same ratio as the price p, that has to be paid today relative to the price 
that has to be paid in the future, also appropriately discounted to its present value price 
(1+ ry) p,41- This interpretation is identical to the two-period interpretation in (2.2.6) 
in ch. 2.2.2. If we normalize prices to unity, (3.1.6) is just the expression we obtained in 
the solution for the two-period maximization problem in (2.6.1). 


3.2 The envelope theorem 


We saw how the Lagrangian can be used to solve optimization problems with many 
time periods. In order to understand how dynamic programming works, it is useful to 
understand a theorem which is frequently used when employing the dynamic programming 
method: the envelope theorem. 
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3.2.1 The theorem 


In general, the envelope theorem says 


Theorem 3.2.1 Let there be a function g(x,y). Choose x such that g(.) is maximized 
for a given y. (Assume g(.) is such that a smooth interior solution exists.) Let f(y) be 
the resulting function of y, 


fy= max g (x,y). 


Then the derivative of f with respect to y equals the partial derivative of g with respect to 
y, if g is evaluated at that x = z(y) that maximizes g, 


d f(y)  Og(z,y) 
dy — ðy 


r=ax(y) 


Proof. f(y) is constructed by 2g (x,y) = 0. This implies a certain x = z (y), 
provided that second order conditions hold. Hence, f (y) = max, g (x,y) = g(x(y),y). 


Then af) — Ga(xly)y) dely) + 09°) The first term of the first term is zero. m 
’ y Ox dy Oy 


3.2.2 Illustration 


The plane depicts the function g (x,y). The maximum of this function with respect to x 
is shown as max,g (x, y), which is f (y). Given this figure, it is obvious that the derivative 
of f (y) with respect to y is the same as the partial derivative of g(.) with respect to y 
at the point where g(.) has its maximum with respect to x: The partial derivative a 
is the derivative when “going in the direction of y”. Choosing the highest point of g (.) 
with respect to x, this directional derivative must be the same as ia at the back of the 


figure. 
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Figure 3.2.1 Illustrating the envelope theorem 


3.2.3 An example 


There is a central planner of an economy. The social welfare function is given by U (A, B), 
where A and B are consumption goods. The technologies available for producing these 
goods are A = A (cLa) and B = B (Lp). The amount of labour used for producing one 
or the other good is denoted by L4 and Lpg and c is a productivity parameter in sector 
A. The economy’s resource constraint is La + Lg = L. 

The planner is interested in maximizing the social welfare level and allocates labour 
according to maxz, U (A (cLa), B (L — La)). The optimality condition is 


— A'c— Basi (3.2.1) 
This makes optimal employment L4 a function of c, 
La = La (c) i (3.2.2) 


The central planner now asks what happens to the social welfare level when the tech- 
nology parameter c increases and she still maximizes the social welfare. The latter requires 
that (3.2.1) continues to hold and the maximized social welfare function with (3.2.2) and 
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the resource constraint reads U (A (cLa (c)), B (L — La (c))). Without using the envelope 
theorem, the answer is 


dy (A (cLa (c)), B(L — La (c))) 


dc 
aO OB ri) = PATAI > 0, 


= aA [La (c) + cL’, (c)] + 


where the last equality follows from inserting the optimality condition (3.2.1). Economi- 
cally, this result means that the effect of better technology on overall welfare is given by 
the direct effect in sector A. The indirect effect through the reallocation of labour vanishes 
as, due to the first-order condition (3.2.1), the marginal contribution of each worker is 
identical across sectors. Clearly, this only holds for a small change in c. 

If one is interested in finding an answer by using the envelope theorem, one would 
start by defining a function V (c) = maxz, U (A (cLa), B (L — La)). Then, according to 
the envelope theorem, 


d o 


—V (c) = —U(A (cLa) ,B (L = La)) 
de de La=La(c) 
OU (.) a OU Cys: 
= a A Tie (S240 


Apparently, both approaches yield the same answer. Applying the envelope theorem gives 
the answer faster. 


3.3 Solving by dynamic programming 


3.3.1 The setup 


We will now get to know how dynamic programming works. Let us study a maximization 
problem which is similar to the one in ch. 3.1.1. We will use the same utility function as in 
(3.1.1), reproduced here for convenience, U, = U°,87 ‘u(c-). The constraint, however, 
will be represented in a more general way than in (3.1.3). We stipulate that there is a 
variable x; which evolves according to 


Tt+1 = hi (Xt, Ct) $ (3.3.1) 


This variable x; could represent wealth and this constraint could then represent the budget 
constraint of the household. This difference equation could also be non-linear, however, 
as for example in a central planner problem where the constraint is a resource constraint 
as in (3.9.3). In this case, x; would stand for capital. Another standard example for x; as 
a state variable would be environmental quality. Here we will treat the general case first 
before we go on to more specific examples further below. 
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The consumption level c; and - more generally speaking - other variables whose value 
is directly chosen by individuals, e.g. investment levels or shares of wealth held in different 
assets, are called control variables. Variables which are not under the direct control of 
individuals are called state variables. In many maximization problems, state variables 
depend indirectly on the behaviour of individuals as in (3.3.1). State variables can also 
be completely exogenous like for example the TFP level in an exogenous growth model 
or prices in a household maximization problem. 

Optimal behaviour is defined by max;.,} U; subject to (3.3.1), ie. the highest value 
U, can reach by choosing a sequence {c,} = {c, ¢41,...} and by satisfying the constraint 
(3.3.1). The value of this optimal behaviour or optimal program is denoted by 


V (x) = ma U, subject to t141 = f (Xtc). (3.3.2) 


V (x+) is called the value function. It is a function of the state variable x, and not of the 
control variable c;. The latter point is easy to understand if one takes into account that 
the control variable c; is, when behaving optimally, a function of the state variable x+. 

The value function V (.) could also be a function of time t (e.g. in problems with finite 
horizon). Generally speaking, x; and c; could be vectors and time could then be part of 
the state vector x+. The value function is always a function of the states of the system or 
of the maximization problem. 


3.3.2 Three dynamic programming steps 


Given this description of the maximization problem, solving by dynamic programming 
essentially requires us to go through three steps. This three-step approach will be followed 
here, later in continuous time, and also in models with uncertainty. 


e DP1: Bellman equation and first-order conditions 


The first step establishes the Bellman equation and computes first-order conditions. 
The objective function U; in (3.1.1) is additively separable which means that it can be 
written in the form 


U: = ufc) + BU 441. (3.3.3) 


Bellman’s idea consists of rewriting the maximization problem in the optimal program 
(3.3.2) as 
V (x) = es {u(cr) + BV (xe41)} (3.3.4) 


subject to 
Ti = f (£i, Ct). 
Equation (3.3.4) is known as the Bellman equation. In this equation, the problem 


with potentially infinitely many control variables {c,} was broken down in many problems 
with one control variable œ. Note that there are two steps involved: First, the additive 
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separability of the objective function is used. Second, more importantly, U;,; is replaced 
by V(a+41). This says that we assume that the optimal problem for tomorrow is solved 
and we should worry about the maximization problem of today only. 

We can now compute the first-order condition which is of the form 


Of (Xt, Ct) 


w (cy) + BV’ (2441) ac, 


=; (3.3.5) 
It tells us that increasing consumption c; has advantages and disadvantages. The advan- 
tages consist in higher utility today, which is reflected here by marginal utility u’ (c+) . The 
disadvantages come from lower overall utility - the value function V - tomorrow. The re- 
duction in overall utility amounts to the change in 211, i.e. the derivative Of (x4, c+) /Oc:, 
times the marginal value of 7414, i.e. V’ (2441). As the disadvantage arises only tomorrow, 
this is discounted at the rate 6. One can talk of a disadvantage of higher consumption 
today on overall utility tomorrow as the derivative Of (x+, c+) /Oc, needs to be negative, 
otherwise an interior solution as assumed in (3.3.5) would not exist. 

In principle, this is the solution of our maximization problem. Our control variable 
& is by this expression implicitly given as a function of the state variable, c; = c (x+), as 
Xt41 by the constraint (3.3.1) is a function of x, and c. As all state variables in t are 
known, the control variable is determined by this optimality condition. Hence, as V is 
well-defined above, we have obtained a solution. 

As we know very little about the properties of V at this stage, however, we need to go 
through two further steps in order to eliminate V (to be precise, its derivative V’ (2441), 
i.e. the costate variable of x;,,) from this first-order condition and obtain a condition 
that uses only functions (like e.g. the utility function or the technology for production in 
later examples) of which properties like signs of first and second derivatives are known. 
We obtain more information about the evolution of this costate variable in the second 
dynamic programming step. 


e DP2: Evolution of the costate variable 


The second step of the dynamic programming approach starts from the “maximized 
Bellman equation”. The maximized Bellman equation is obtained by replacing the control 
variables in the Bellman equation, i.e. the c in (3.3.4), in the present case, by the optimal 
control variables as given by the first-order condition, i.e. by c (x+) resulting from (3.3.5). 
Logically, the max operator disappears (as we insert the c(z;) which imply a maximum) 
and the maximized Bellman equation reads 


V (x) = u(c(ae)) + BV (f (ae, e(@e))) - 
The derivative with respect to x; reads 


dc (xz) 


V (ay) = (e) SS 


+ BV (F (ae (00) [fn + 2. 


dx 
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This step shows why it is important that we use the maximized Bellman equation here: 
Now control variables are a function of state variable and we need to compute the deriv- 
ative of œ with respect to x, when computing the derivative of the value function V (x+). 
Inserting the first-order condition simplifies this equation to 


V" (x1) = BV" (F (at, ce (21))) for = BV’ (£41) for (3.3.6) 


This equation is a difference equation for the costate variable, the derivative of the 
value function with respect to the state variable, V’(z,). The costate variable is also 
called the shadow price of the state variable x;. If we had more state variables, there 
would be a costate variable for each state variable. It says how much an additional unit 
of the state variable (say e.g. of wealth) is valued: As V (x+) gives the value of optimal 
behaviour between t and the end of the planning horizon, V’ (x+) says by how much this 
value changes when 2; is changed marginally. Hence, equation (3.3.6) describes how the 
shadow price of the state variable changes over time when the agent behaves optimally. 
If we had used the envelope theorem, we would have immediately ended up with (3.3.6) 
without having to insert the first-order condition. 


e DP3: Inserting first-order conditions 


Now insert the first-order condition (3.3.5) twice into (3.3.6) to replace the unknown 
shadow price and to find an optimality condition depending on u, only. This will then be 
the Euler equation. We do not do this here explicitly as many examples will go through 
this step in detail in what follows. 


3.4 Examples 


3.4.1 Intertemporal utility maximization with a CES utility func- 
tion 
The individual’s budget constraint is given in the dynamic formulation 


a1 = (1+ r4) (a, + w — &)- (3.4.1) 


Note that this dynamic formulation corresponds to the intertemporal version in the sense 
that (3.1.3) implies (3.4.1) and (3.4.1) with some limit condition implies (3.1.3). This will 
be shown formally in ch. 3.5.1. 

The budget constraint (3.4.1) can be found in many papers and also in some textbooks. 
The timing as implicit in (3.4.1) is illustrated in the following figure. All events take place 
at the beginning of the period. Our individual owns a certain amount of wealth a; at the 
beginning of t and receives here wage income w; and spends cj on consumption also at 
the beginning. Hence, savings s; can be used during t for production and interest is paid 
on s; which in turn gives aşı at the beginning of period t + 1. 
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St = Ay + Wy — Cr ary = (1+7 Js, 


Ct Ct+1 


| i | 


t+1 


Figure 3.4.1 The timing in an infinite horizon discrete time model 


The consistency of (3.4.1) with technologies in general equilibrium is not self-evident. 
We will encounter more conventional budget constraints of the type (2.5.13) further below. 
As (3.4.1) is widely used, however, we now look at dynamic programming methods and 
take this budget constraint as given. 

The objective of the individual is to maximize her utility function (3.1.1) subject to the 
budget constraint by choosing a path of consumption levels c,, denoted by {c,},7 € [t, co]. 
We will first solve this with a general instantaneous utility function and then insert the 


CES version of it, i.e. 
Ces (3.4.2) 
L=@ 
The value of the optimal program {c+} is, given its initial endowment with a;, defined as 
the maximum which can be obtained subject to the constraint, i.e. 


V (a) = a U (3.4.3) 
subject to (3.4.1). It is called the value function. Its only argument is the state variable 
az. See ch. 3.4.2 for a discussion on state variables and arguments of value functions. 


e DP1: Bellman equation and first-order conditions 


We know that the utility function can be written as U; = u (c+) + GUz41. Now assume 
that the individual behaves optimally as from t+ 1. Then we can insert the value function. 
The utility function reads U; = u (ct) + BV (at41). Inserting this into the value function, 
we obtain the recursive formulation 


V (ae) = max {u (ce) + BV (ae41)}, (3.4.4) 


known as the Bellman equation. 

Again, this breaks down a many-period problem into a two-period problem: The 
objective of the individual was max,,,} (3.1.1) subject to (3.4.1), as shown by the value 
function in equation (3.4.3). The Bellman equation (3.4.4), however, is a two period 
decision problem, the trade-off between consumption today and more wealth tomorrow 
(under the assumption that the function V is known). This is what is known as Bellman’s 
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principle of optimality: Whatever the decision today, subsequent decisions should be made 
optimally, given the situation tomorrow. History does not count, apart from its impact 
on the state variable(s). 

We now derive a first-order condition for (3.4.4). It reads 


d 
dc, 


d ; d 
u (ci) + BTV (aes) = u (cx) + BV" (a) FE =O 


Since da;41/dc, = — (1 + r;) by the budget constraint (3.4.1), this gives 
u (cr) — 81 +1) V’ (a1) = 0. (3.4.5) 


Again, this equation makes consumption a function of the state variable, c = c (at) . 
Following the first-order condition (3.3.5) in the general example, we wrote c; = c(2;), 
i.e. consumption c; changes only when the state variable x, changes. Here, we write 
C = G (az) , indicating that there can be other variables which can influence consumption 
other than wealth a;. An example for such an additional variable in our setup would be 
the wage rate w; or interest rate r+, which after all is visible in the first-order condition 
(3.4.5). See ch. 3.4.2 for a more detailed discussion of state variables. 

Economically, (3.4.5) tells us as before in (3.3.5) that, under optimal behaviour, gains 
from more consumption today are just balanced by losses from less wealth tomorrow. 
Wealth tomorrow falls by 1 + r+, this is evaluated by the shadow price V’ (a1) and 
everything is discounted by 6. 


e DP2: Evolution of the costate variable 


Using the envelope theorem, the derivative of the maximized Bellman equation reads, 


ONTAS 


Oat l 


V" (at) = BV" (ae41) (3.4.6) 


We compute the partial derivative of a;,, with respect to a; as the functional relationship 
of & = c (a+) should not (because of the envelope theorem) be taken into account. See 
exercise 2 on p. 71 on a derivation of (3.4.6) without using the envelope theorem. 

From the budget constraint we know that wo = 1 + r;. Hence, the evolution of the 
shadow price/ the costate variable under optimal behaviour is described by 


V' (at) = P [1 + ra] V” (att). 
This is the analogon to (3.3.6). 
e DP3: Inserting first-order conditions 


Let us now be explicit about how to insert first-order conditions into this equation. We 
can insert the first-order condition (3.4.5) on the right-hand side. We can also rewrite the 
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first-order condition (3.4.5), by lagging it by one period, as 8 (1 + r1) V” (ar) = u’ (c1) 
and can insert this on the left-hand side. This gives 


w (1) BT ra) =u (a) S u (a) = BL rdu lera). (3.4.7) 


This is the same result as the one we obtained when we used the Lagrange method in 
equation (3.1.6). 

It is also the same result as for the two-period saving problem which we found in 
OLG models - see e.g. (2.2.6) or (2.6.1) in the exercises. This might be surprising as 
the planning horizons differ considerably between a two- and an infinite-period decision 
problem. Apparently, whether we plan for two periods or for many more, the change 
between two periods is always the same when we behave optimally. It should be kept in 
mind, however, that consumption levels (and not changes) do depend on the length of the 
planning horizon. 


e The CES and logarithmic version of the Euler equation 


Let us now insert the CES utility function from (3.4.2) into (3.4.7). Computing mar- 
ginal utility gives u’ (c+) = c77 and we obtain a linear difference equation in consumption, 


C1 = (BL +r)? ce. (3.4.8) 


Note that the logarithmic utility function u(c,) = Inc,, known for the two-period 
setup from (2.2.1), is a special case of the CES utility function (3.4.2). Letting o approach 
unity, we obtain 


ge = 1 
lim u (c,) = lim 4+—— = Inc, 
onl o—>l1 1 — 0 


where the last step used L’Héspital’s rule: The derivative of the numerator with respect 
to ø is 


d d 
ae (9 1) = aS (erence =1)= e=) mner (Ine) =e"? (— Ine). 
o o 
Hence, 
crete? (-Inc,) 
lim = = lim =< Imes. (3.4.9) 


When the logarithmic utility function is inserted into (3.4.7), one obtains an Euler equa- 
tion as in (3.4.8) with o set equal to one. 


3.4.2 What is a state variable? 


Dynamic programming uses the concept of a state variable. In the general version of 
ch. 3.3, there is clearly only one state variable. It is z; and its evolution is described in 
(3.3.1). In the economic example of ch. 3.4.1, the question of what is a state variable is 
less obvious. 
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In a strict formal sense, “everything” is a state variable. “Everything” means all 
variables which are not control variables are state variables. This very broad view of state 
variables comes from the simple definition that “everything (apart from parameters) which 
determines control variables is a state variable”. 


We can understand this view by looking at the explicit solution for the control variables 
in the two-period example of ch. 2.2.1. We reproduce (2.2.3), (2.2.4) and (2.2.5) for ease 
of reference, 


We = wet (1 trey) west, 
G41 = (L-y) (1+ ra) Wi 
G = YW. 


We did not use the terms control and state variable there but we could of course solve 
this two-period problem by dynamic programming as well. Doing so would allow us to 
understand why “everything” is a state variable. Looking at the solution for c;,, shows 
that it is a function of r41, wp and w1. If we want to make the statement that the 
control variable is a function of the state variables, then clearly 7441, we and w1 are 
state variables. 


Generalizing this for our multi-period example from ch. 3.4.1, the entire paths of r, and 
w, are state variables, in addition to wealth a,. As we are in a deterministic world, we 
know the evolution of variables r, and w, and we can reduce the path of r; and w; by the 
levels of r, and w, plus the parameters of the process describing their evolution. Hence, 
the broad view for state variables applied to ch. 3.4.1 requires us to use r, w;, a; as state 
variables. 


This broad (and ultimately correct) view of state variables is the reason why the first- 
order condition (3.4.5) is summarized by c = c (a+). The index t captures all variables 
which influence the solution for c apart from the explicit argument a+. 


In a more practical sense - as opposed to the strict sense - it is highly recommended 
to consider only the variable which is indirectly affected by the control variable as (the 
relevant) state variable. Writing the value function as V = V (a, wą r+) is possible but 
highly cumbersome from a notational point of view. What is more, going through the 
dynamic programming steps does not require more than a; as a state variable as only the 
shadow price of a; is required to obtain an Euler equation and not the shadow price of w 
or r;. To remind us that more than just a; has an impact on optimal controls, we should, 
however, always write c = c (a;) as a shortcut for c = € (at, Wi, Tt, --) - 


The conclusion of all this, however, is more cautious: When encountering a new max- 
imization problem and when there is uncertainty about how to solve it and what is a 
state variable and what is not, it is always the best choice to include more rather than 
less variables as arguments of the value function. Dropping some arguments afterwards 
is simpler than adding additional ones. 
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3.4.3 Optimal R&D effort 


In this second example, a research project has to be finished at some future known point 
in time T. This research project has a certain value at point T and we denote it by D 
like dissertation. In order to reach this goal, a path of a certain length L needs to be 
completed. We can think of L as a certain number of pages, a certain number of papers 
or - probably better - a certain quality of a fixed number of papers. Going through this 
process of walking and writing is costly, it requires effort e, at each point in time T > t. 
Summing over these cost - think of them as hours worked per day - eventually yields the 
desired amount of pages, 

eel (ee, (3.4.10) 


where f(.) is the page of quality production function: More effort means more output, 
f’(.) > 0, but any increase in effort implies a lower increase in output, f”(.) < 0. 
The objective function of our student is given by 


U; = 8D -DI 6e. (3.4.11) 


The value of the completed dissertation is given by D and its present value is obtained by 
discounting at the discount factor 3. Total utility U; stems from this present value minus 
the present value of research cost e,. The maximization problem consists in maximizing 
(3.4.11) subject to the constraint (3.4.10) by choosing an effort path {e,}. The question 
now arises how these costs are optimally spread over time. How many hours should be 
worked per day? 

An answer can be found by using the Lagrange-approach with (3.4.11) as the objective 
function and (3.4.10) as the constraint. However, her we will use the dynamic program- 
ming approach. Before we can apply it, we need to derive a dynamic budget constraint. 
We therefore define 

M; = ee) 


as the amount of the pages that have already been written by today. This then implies 
Misi = Mı = fle). (3.4.12) 


The increase in the number of completed pages between today and tomorrow depends on 
effort-induced output f (e,) today. We can now apply the three dynamic programming 
steps. 


e DP1: Bellman equation and first-order conditions 


The value function can be defined by V (M;) = max; U; subject to the constraint. 
We follow the approach discussed in ch. 3.4.2 and explicitly use as state variable M, only, 
the only state variable relevant for derivations to come. In other words, we explicitly 
suppress time as an argument of V (.). The reader can go through the derivations by 
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using a value function specified as V (M;, t) and find out that the same result will obtain. 
The objective function U; written recursively reads 


U, =6 (ae = ee eae $ aD ~B(e,4 T aS) 


= 8 (Br D — SF Be, | — p 


Assuming that the individual behaves optimally as from tomorrow, this reads U; = —e;+ 
BV (Mp1) and the Bellman equation reads 
V (M) = max {—e + BV (Mi41)} š (3.4.13) 


The first-order condition is —1 + 6V'(Mi+ı )wet = 0, which, using the dynamic 
budget constraint, becomes 


1 = BV! (Mess) f’ (e1). (3.4.14) 


Again as in (3.3.5), implicitly and with (3.4.12), this equation defines a functional re- 
lationship between the control variable and the state variable, ep = e (Mz). One unit 
of additional effort reduces instantaneous utility by 1 but increases the present value of 
overall utility tomorrow by BV’ (M41) f’ (ez). 


e DP2: Evolution of the costate variable 


To provide some variation, we will now go through the second step of dynamic pro- 
gramming without using the envelope theorem. Consider the maximized Bellman equa- 
tion, where we insert e, = e (M+) and (3.4.12) into the Bellman equation (3.4.13), 


V (Mi) = —e (Mi) + BV (M: + fle (M)))- 


The derivative with respect to Mi is 


V' (Mi) = —e (Mi) + BV" (M, + fle (M) a “ie a 


= —e' (Mz) + BV’ (Misi) [L + f (e (Mi) e (Ma). 


Using the first-order condition (3.4.14) simplifies this derivative to V’ (M+) = BV’ (Mi41). 
Expressed for t + 1 gives 


V" (Mis) = BV"(Mi+2) (3.4.15) 
e DP3: Inserting first-order conditions 


The final step inserts the first-order condition (3.4.14) twice to replace V’ (M;.;) and 
Vv’ (Mi+2) ’ 


-ly pI “iite i f'(€t41) 
BF le) = (Feta) Pler 


The interpretation of this Euler equation is now simple. As f”(.) < 0 and 8 < 1, effort 
e, increases under optimal behaviour, i.e. e1 > e. Optimal writing of a dissertation 
implies more work every day. 


= B. (3.4.16) 
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e What about levels? 


The optimality condition in (3.4.16) specifies only how effort e, changes over time, it 
does not provide information on the level of effort required every day. This is a property 
of all expressions based on first-order conditions of intertemporal problems. They only 
give information about changes of levels, not about levels themselves. However, the basic 
idea for how to obtain information about levels can be easily illustrated. 

Assume f (e;) = e}, with 0 < y < 1. Then (3.4.16) implies (with t being replaced by 
T) ey / 1 =8 Se, = 6/9 Ve,. Solving this difference equation yields 


e, = 8700-e, (3.4.17) 


where e; is the (at this stage still) unknown initial effort level. Starting in 7 = 1 on the 
first day, inserting this solution into the intertemporal constraint (3.4.10) yields 


Nf (pe, = DTe (r-1)7/(1 Der = 


This gives us the initial effort level as (the sum can be solved by using the proofs in 


ch. 2.5.1) 
L 1/7 
ne (= C= =) i 


With (3.4.17), we have now also computed the level of effort every day. 

Behind these simple steps, there is a general principle. Modified first-order conditions 
resulting from intertemporal problems are difference equations, see for example (2.2.6), 
(3.1.6), (3.4.7) or (3.4.16) (or differential equations when we work in continuous time 
later). Any difference (or also differential) equation when solved gives a unique solution 
only if an initial or terminal condition is provided. Here, we have solved the difference 
equation in (3.4.16) assuming some initial condition e1. The meaningful initial condition 
then followed from the constraint (3.4.10). Hence, in addition to the optimality rule 
(3.4.16), we always need some additional constraint which allows us to compute the level 
of optimal behaviour. We return to this point when looking at problems in continuous 
time in ch. 5.4. 


3.5 On budget constraints 


We have encountered two different (but related) types of budget constraints so far: dy- 
namic ones and intertemporal ones. Consider the dynamic budget constraint derived in 
(2.5.13) as an example. Using e; = p,c; for simplicity, it reads 


Aes. = (1+ ri) ai + Wi — er (3.5.1) 


This budget constraint is called dynamic as it “only” takes what happens between the two 
periods t and t + 1,into account. In contrast, an intertemporal budget constraint takes 


60 Chapter 3. Multi-period models 


what happens between any starting period (usually t) and the end of the planning horizon 
into account. In this sense, the intertemporal budget constraint is more comprehensive 
and contains more information (as we will also see formally below whenever we talk about 
the no-Ponzi game condition). An example for an intertemporal budget constraint was 
provided in (3.1.3), replicated here for ease of reference, 


Tse! Veo a aay ae, (3.5.2) 


3.5.1 From intertemporal to dynamic 


We will now ask about the link between dynamic and intertemporal budget constraints. 
Let us choose the simpler link to start with, i.e. the link from the intertemporal to the 
dynamic version. As an example, take (3.5.2). We will now show that this intertemporal 
budget constraint implies 


Qt41 = (1 oe r) (a + We - Ct) , (3.5.3) 


which was used before, for example in (3.4.1). 
Write (3.5.2) for the next period as 


Seay haere) ey eed i ae (3.5.4) 
Express (3.5.2) as 


ert Darga (Lt ee er = Oy + Wi + OP, (1+ ome Wr = 
eg + (1+ ry! Dep (1 F DaD. er = a +w + (14 ry Era leat ry ey) w, & 
EL (Lt ry e, = (Ltr) (ae + w — ei) + ERa (Dry w. 


Insert (3.5.4) and find the dynamic budget constraint (3.5.3). 


3.5.2 From dynamic to intertemporal 


Let us now ask about the link from the dynamic to the intertemporal budget constraint. 
How can we obtain the intertemporal version of the budget constraint (3.5.1)? 

Technically speaking, this simply requires us to solve a difference equation: In order 
to solve (3.5.1) recursively, we rewrite it as 


Pas o f O tpi + Cti — Witi 
$=) p= . 
I1+r i; 14+ Tihi 
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Inserting sufficiently often yields 


nE Meo eiu et — Wt = Qt+2 + Cr41 — Wt41 | Ct — Ut 
1+r (1+ra) (+r eas 
ee Cop — Wii a Ut 
(1+ ri) (1 +172) ltr 
E At43 + Et+2 — Wee Ct41 — Wii _ ee Wt 
“Centon an) Cae Tae 
Atti os Ct+i — Weta 


2 cen 3s 
i>oo (1+ Tei) (l tre) (ltr) O (L+ rii) (+ rea) (L + re) 


The expression in the last line is hopefully instructive but somewhat cumbersome. We 
can write it in a more concise way as 


Qt+i oyo _ St+i — Weti 
F Ai=0 77i 
Mi-o (1 + fis) 


where II indicates a product, i.e. Ii o (1+ ri) = 1+ r; for i = 0 and I’ o (1+ Ti) = 
(1+ rii) (1+ r) for i > 0. For i = —1, Io (1 + fi) = 1 by definition. 
Letting the limit be zero, a step explained in a second, we obtain 


at = lim i= 
EPS Tia (1 F Tits) 


O o a =D 
ae Iso (1+ Tits) a ih Sa (1+ ris) a R, 


a Dee ee ore =e 


where the last but one equality is substituted t + 7 by r. We can write this as 


œo Wr 


me T mE 


(3.5.5) 


T=t R, z i R- 
With a constant interest rate, this reads 
Ee e esn EE S aie, (3.5.6) 


Equation (3.5.5) is the intertemporal budget constraint that results from a dynamic bud- 
get constraint as specified in (3.5.1) using the additional condition that 
lim; ort = 

Note that the assumption that this limit is zero has a standard economic interpreta- 
tion. It is usually called the no-Ponzi game condition. To understand the interpretation 
more easily, just focus on the case of a constant interest rate. The condition then reads 
liM; @¢4i/ (1+ 7r)' = 0. The term a;;;/(1+7)' is the present value in t of wealth atp; 
held in t + i. The condition says that this present value must be zero. 

Imagine an individual that finances expenditure e, by increasing debt, i.e. by letting 
Qipi becoming more and more negative. This condition simply says that an individual’s 
“long-run” debt level, i.e. a; for i going to infinity must not increase too quickly - 
the present value must be zero. Similarly, the condition also requires that an individual 
should not hold positive wealth in the long run whose present value is not zero. Note that 
this condition is fulfilled , for example, for any constant debt or wealth level. 
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3.5.3 Two versions of dynamic budget constraints 


Note that we have also encountered two subspecies of dynamic budget constraints. The 
one from (3.5.1) and the one from (3.5.3). The difference between these two constraints is 
due to more basic assumptions about the timing of events as was illustrated in fig.s 2.1.1 
and 3.4.1. 

These two dynamic constraints imply two different versions of intertemporal budget 
constraints. The version from (3.5.1) leads to (3.5.6) and the one from (3.5.3) leads (with 
a similar no-Ponzi game condition) to (3.5.2). Comparing (3.5.6) with (3.5.2) shows that 
the present values on both sides of (3.5.6) discounts one time more than in (3.5.2). The 
economic difference again lies in the timing, i.e. whether we look at values at the beginning 
or end of a period. 

The budget constraint (3.5.1) is the “natural” budget constraint in the sense that it can 
be derived easily as above in ch. 2.5.5 and in the sense that it easily aggregates to economy 
wide resource constraints. We will therefore work with (3.5.1) and the corresponding 
intertemporal version (3.5.6) in what follows. The reason for not working with them right 
from the beginning is that the intertemporal version (3.5.2) has some intuitive appeal and 
that its dynamic version (3.5.3) is widely used in the literature. 


3.6 <A decentralized general equilibrium analysis 


We have so far analyzed maximization problems of households in partial equilibrium. In 
two-period models, we have analyzed how households can be aggregated and what we 
learn about the evolution of the economy as a whole. We will now do the same for infinite 
horizon problems. 

As we did in ch. 2.4, we will first specify technologies. This shows what is technologi- 
cally feasible in this economy. Which goods are produced, which goods can be stored for 
saving purposes, is there uncertainty in the economy stemming from production processes? 
Given these technologies, firms maximize profits. Second, household preferences are pre- 
sented and the budget constraint of households is derived from the technologies presented 
before. This is the reason why technologies should be presented before households are 
introduced: budget constraints are endogenous and depend on knowledge of what house- 
holds can do. Optimality conditions for households are then derived. Finally, aggregation 
over households and an analysis of properties of the model using the reduced form follows. 


3.6.1 Technologies 
The technology is a simple Cobb-Douglas technology 
Y, = AKEL". (3.6.1) 


Capital K, and labour L is used with a given total factor productivity level A to produce 
output Y;. This good can be used for consumption and investment and equilibrium on the 
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goods market requires 
Y= Cith. (3.6.2) 


Gross investment J; is turned into net investment by taking depreciation into account, 
K1 = (1 — ô) Ki + L. Taking these two equations together gives the resource constraint 
of the economy, 

Kui = (1-6) Ki +Y; — Ci. (3.6.3) 


As this constraint is simply a consequence of technologies and market clearing, it is 
identical to the one used in the OLG setup in (2.4.9). 


3.6.2 Firms 


Firms maximize profits by employing optimal quantities of labour and capital, given the 
technology in (3.6.1). First-order conditions are 


OY; OY, 
aK, =—wk, =w} (3.6.4) 


as in (2.4.2), where we have again chosen the consumption good as numéraire. 


3.6.3 Households 


Preferences of households are described as in the intertemporal utility function (3.1.1). 
As the only way households can transfer savings from one period to the next is by buying 
investment goods, an individual’s wealth is given by the “number of machines” k+, she 
owns. Clearly, adding up all individual wealth stocks gives the total capital stock, Upk, = 
K. Wealth k; increases over time if the household spends less on consumption than what 
it earns through capital plus labour income, corrected for the loss in wealth each period 
caused by depreciation, 


kiya — ky = wp ky — Ok, + we — c eg = (L+ uf — 8) ki + wy — c 
If we now define the interest rate to be given by 
r= wk —6, (3.6.5) 
we obtain our budget constraint 
kor = (1 + ri) ki + wl — c. (3.6.6) 


Note that the “derivation” of this budget constraint was simplified in comparison to 
ch. 2.5.5 as the price v; of an asset is, as we measure it in units of the consumption 
good which is traded on the same final market (3.6.2), given by 1. More general budget 
constraints will become pretty complex as soon as the price of the asset is not normalized. 
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This complexity is needed when it comes e.g. to capital asset pricing - see further below 
in ch. 9.3. Here, however, this simple constraint is perfect for our purposes. 

Given the preferences and the constraint, the Euler equation for this maximization 
problem is given by (see exercise 5) 


u (ct) = B [1 + repi] wu’ (Cipa). (3.6.7) 


Structurally, this is the same expression as in (3.4.7). The interest rate, however, refers 
to t + 1, due to the change in the budget constraint. Remembering that 6 = 1/ (1 + p), 
this shows that consumption increases as long as r441 > p. 


3.6.4 Aggregation and reduced form 

e Aggregation 

To see that individual constraints add up to the aggregate resource constraint, we 
simply need to take into account that individual income adds up to output, wi Ki+wt L; = 


Y;. Remember that we are familiar with the latter from (2.4.4). Now start from (3.6.6) 
and use (3.6.5) to obtain, 


ha = Yrki = (1+ w7* — ô) Erki + wL- C, = (1 — 8) Ki + Yı — Ci. 

The optimal behaviour of all households taken together can be gained from (3.6.7) 
by summing over all households. This is done analytically correctly by first applying the 
inverse function of u’ to this equation and then summing individual consumption levels 
over all households (see exercise 6 for details). Applying the inverse function again gives 


wl (Ci) = B [1 + rea] e (Ciri), (3.6.8) 


where C, is aggregate consumption in t. 

e Reduced form 

We now need to understand how our economy evolves in general equilibrium. Our first 
equation is (3.6.8), telling us how consumption evolves over time. This equation contains 
consumption and the interest rate as endogenous variables. 


Our second equation is therefore the definition of the interest rate in (3.6.5) which we 
combine with the first-order condition of the firm in (3.6.4) to yield 


n= 6. (3.6.9) 


This equation contains the interest rate and the capital stock as endogenous variables. 


3.6. A decentralized general equilibrium analysis 65 


Our final equation is the resource constraint (3.6.3), which provides a link between 
capital and consumption. Hence, (3.6.8), (3.6.9) and (3.6.3) give a system in three equa- 
tions and three unknowns. When we insert the interest rate into the optimality condition 
for consumption, we obtain as our reduced form 


1 _ OY¥t41 1 
ul (Cy) = B [1 + Se — 5] wl (Cras), Aue 
Kuss = (1 — ô) Ki +Y; — Ci 
This is a two-dimensional system of non-linear difference equations which gives a unique 
solution for the time path of capital and consumption, provided we have two initial con- 
ditions Ko and Co. 


3.6.5 Steady state and transitional dynamics 


When trying to understand a system like (3.6.10), the same principles can be followed as 
with one-dimensional difference equations. First, one tries to identify a fixed point, i.e. a 
steady state, and then one looks at transitional dynamics. 


e Steady state 


In a steady state, all variables are constant. Setting Kı = Ki = K and Chi = Gi = 
C, we obtain 


OY OY 
ie 1+—-é6]s-=>= 6, C=Y-6K 
where the “<> step” used the link between 6 and p from (3.1.2). In the steady state, the 
marginal productivity of capital is given by the time preference rate plus the depreciation 
rate. Consumption equals output minus depreciation, i.e. minus replacement investment. 
These two equations determine two variables K and C: the first determines K, the second 


determines C. 
e Transitional dynamics 


Understanding transitional dynamics is not as straightforward as understanding the 
steady state. Its analysis follows the same idea as in continuous time, however, and we 
will analyze transitional dynamics in detail there. 

Having said this, we should acknowledge the fact that transitional dynamics in discrete 
time can quickly become more complex than in continuous time. As an example, chaotic 
behaviour can occur in one-dimensional difference equations while one needs at least a 
three-dimensional differential equation system to obtain chaotic properties in continuous 
time. The literature on chaos theory and textbooks on difference equations provide many 
examples. 
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3.7 A central planner 


3.7.1 Optimal factor allocation 


One of the most solved maximization problems in Economics is the central planner prob- 
lem. The choice by a central planner given a social welfare function and technological 
constraints provides information about the first-best factor allocation. This is a bench- 
mark for many analyses in normative economics. We consider the probably most simple 
case of optimal factor allocation in a dynamic setup. 


e The maximization problem 


Let preferences be given by 
U= p ule (3.7.1) 


where C, is the aggregate consumption of all households at a point in time 7. This function 
is maximized subject to a resource accumulation constraint which reads 


Kap Vii bay Pak Se. (3.7.2) 


for all r > t. The production technology is given by a neoclassical production function 
Y (K,, £,) with standard properties. 


e The Lagrangian 


This setup differs from the ones we got to know before in that there is an infinite 
number of constraints in (3.7.2). This constraint holds for each point in time T > t. As a 
consequence, the Lagrangian is formulated with infinitely many Lagrangian multipliers, 


L = EL, Bu (C) + EL, fA; [Krai — Y (Kz, L,) — (1 — ô) K, + C]}. (3.7.3) 


The first part of the Lagrangian is standard, £% ,867*u (C,), it just copies the objective 
function. The second part consists of a sum from ¢ to infinity, one constraint for each 
point in time, each multiplied by its own Lagrange multiplier A+. In order to make the 
maximization procedure clearer, we rewrite the Lagrangian as 


B= SPB Mi Ce) Oke, [Kea Y (Kh) (1 6) he CC 
+ às-1 [Ks — Y (Ks-1, L-1) — (1 — ô) he + Cs-1] 

+ às ett — Y (Ks, Ls) — = ô) Ks + C] 

He Scam aS a een e sh 


where we simply explicitly write out the sum for s — 1 and s. 
Now maximize the Lagrangian both with respect to the control variable C., the mul- 
tipliers A, and the state variables K,. Maximization with respect to K, might appear 
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unusual at this stage; we will see a justification for this in the next chapter. First-order 
conditions are 


Lo, = Btu (C) +A, =0 S A, = —u' (C,) 8°, (3.7.4) 
OY As—1 OY 
4 = Eee =14 =%, T 
LK, = Às-1 — À ES | < F JK, ð (3.7.5) 
Ly, =0. (3.7.6) 
Combining the first and second first-order condition gives a =14 ae — ô 
This is equivalent to 
u’ (C3) 1 
; = i ; (3.7.7) 
Buw (Cs41) oo, 


This expression has the same interpretation as (3.1.6) or (3.4.7) for example. When we 
replace s by 7, this equation with the constraint (3.7.2) is a two-dimensional difference 
equation system which allows us to determine the paths of capital and consumption, 
given two boundary conditions, which the economy will follow when factor allocation is 
optimally chosen. The steady state of such an economy is found by setting Cs = Cy44 
and K, = Ks+ı in (3.7.7) and (3.7.2). 

This example also allows us to return to the discussion about the link between the 
sign of shadow prices and the Lagrange multiplier at the end of ch. 2.3.2. Here, the 
constraints in the Lagrangian are represented as left-hand side minus right-hand side. As 
a consequence, the Lagrange multipliers are negative, as the first-order conditions (3.7.4) 
show. Apart from the fact that the Lagrange multiplier here now stands for minus the 
shadow price, this does not play any role for the final description of optimality in (3.7.7). 


3.7.2 Where the Lagrangian comes from IT 


Let us now see how we can derive the same expression as that in (3.7.7) without using the 
Lagrangian. This will allow us to give an intuitive explanation for why we maximized the 
Lagrangian in the last chapter with respect to both the control and the state variable. 


e Maximization without Lagrange 
Insert the constraint (3.7.2) into the objective function (3.7.1) and find 


U, = Seer ay (Y (Kr, Ey) F (1 ~~ ô) K, — Kr41) =R Tey 


This is now maximized by choosing a path {K,} for capital. Choosing the state variable 
implicitly pins down the path {C,} of the control variable consumption and one can 


therefore think of this maximization problem as one where consumption is optimally 
chosen. 
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Now rewrite the objective function as 


U; = Xiu (Y (Kr, L,) + (1-5) K, — Kry) 

+ 87t “a (Y (K1, Ls-1) + (1 — 8) Ke-1 — K,) 
+ Ba (Y (Ks, Ls) + (1 = ô) Ks — Ksq) 

+EL 18u (Y (K,,L,) + (1-6) K; — Kra). 


Choosing K, optimally implies 
~ B ti (Y (Ks-1, L1) a (1 = ô) Ks _ K,) 


+ Btu! (Y (Ks, Ls) + (1 — 6) K; — Keys) (Se +1—- 5) =0. 


Reinserting the constraint (3.7.2) and rearranging gives 


we a) 


u (Cs-1) = Be (C5) (1 + JK 


This is the standard optimality condition for consumption which we obtained in (3.7.7). 
As s can stand for any point in time between t and infinity, we could replace s by t, T or 
Ted 


e Back to the Lagrangian 


When we now go back to the maximization procedure where the Lagrangian was used, 
we see that the partial derivative of the Lagrangian with respect to K; in (3.7.5) captures 
how A; changes over time. The simple reason why the Lagrangian is maximized with 
respect to K, is therefore that an additional first-order condition is needed as À; needs to 
be determined as well. 

In static maximization problems with two consumption goods and one constraint, the 
Lagrangian is maximized by choosing consumption levels for both consumption goods and 
by choosing the Lagrange multiplier. In the Lagrange setup above in (3.7.4) to (3.7.6), we 
choose both endogenous variables K, and C; plus the multiplier A; and thereby determine 
optimal paths for all three variables. Hence, it is a technical - mathematical - reason that 
K, is “chosen”: determining three unknowns simply requires three first-order conditions. 
Economically, however, the control variable C; is “economically chosen” while the state 
variable K, adjusts indirectly as a consequence of the choice of C;. 


3.8 Growth of family size 


3.8.1 The setup 


Let us now consider an extension to the models considered so far. Let us imagine there is 
a family consisting of n, members at point in time 7 and let consumption c, per individual 
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family member be optimally chosen by the head of the family. The objective function 
for this family head consists of instantaneous utility u(.) per family member times the 
number of members, discounted at the usual discount factor p, 


Uy = 52 0" “a (c) nr. 


Let us denote family wealth by â+. It is the product of individual wealth times the 
number of family members, a; = na. The budget constraint of the household is then 
given by 

Ôt = (1+ Ti) Ât + ewe — HEC 


Total labour income is given by n; times the wage w, and family consumption is nc. 


3.8.2 Solving by substitution 


We solve this maximization problem by substitution. We rewrite the objective function 
and insert the constraint twice, 


{oe u (c) nr + Bu (Cs) ns + BOT tu (Co41) Nns41 + De 58" ule) n, } 


i E 1 rs) Âs +NsWs — Gs+ 
= BEA u (e)n + Au ( DETA tats âet) p, 
Ns 
2 1l+r Gs41 + Ns+1Ws+1 — Â = 
+ get u (! s+1) es Seinen 2) naa + Espl ‘u (c+) Nr. 
s+1 


Now compute the derivative with respect to G,,,. This gives 


! (£ ae rs) ûs + Ns Ws — sn) Ns 1 (£ + Pad) ûs+1 + Ns4+1Ws4+1 — “= 1 + Pst 
u = Bu Ns+1- 


Ns Ns Ns+1 Ns+1 


When we replace the budget constraint by consumption again and cancel the n, and n,41, 
we obtain 


u (cs) = B [1 + rs41] U (Ce41)- (3.8.1) 


The interesting feature of this rule is that being part of a family whose size n, can change 
over time does not affect the growth of individual consumption c,. It follows the same rule 
as if individuals maximized utility independently of each other and with their personal 
budget constraints. 


3.8.3 Solving by the Lagrangian 


The Lagrangian for this setup with one budget constraint for each point in time requires 
an infinite number of Lagrange multipliers A,, one for each 7. It reads 


LSN Tou (cr) nr + àr [(1 + r+) à- + nrlrwr — nc, — âr+1l} . 
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We first compute the first-order conditions for consumption and hours worked for one 
point in time s, 

o 
=—u (cs) — As = 0, 

De, (cs) 

As discussed in 3.7, we also need to compute the derivative with respect to the state 
variable. It is important to compute the derivative with respect to family wealth â, as 
this is the true state variable of the head of the family. (Computing the derivative with 
respect to individual wealth a, would also work but would lead to an incorrect result, i.e. 
a result that differs from 3.8.1.) This derivative is 


be = ore 


La, = —As-1 + As (1 + rs) = 04 Asa = (1 +175) às. 


Optimal consumption then follows by replacing the Lagrange multipliers, wu’ (c,-1) = 
B [1 +rs] w (cs). This is identical to the result we obtain by inserting in (3.8.1). 


3.9 Further reading and exercises 


For a much more detailed background on the elasticity of substitution, see Blackorby 
and Russell (1989). They study the case of more than two inputs and stress that the 
Morishima elasticity is to be preferred to the Allan/ Uzawa elasticity. 

The dynamic programming approach was developed by Bellman (1957). Maximization 
using the Lagrange method is widely applied by Chow (1997). The example in ch. 3.4.3 
was originally inspired by Grossman and Shapiro (1986). 
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Exercises chapter 3 
Applied Intertemporal Optimization 


Dynamic programming in discrete deterministic time 


1. The envelope theorem I 
Let the utility function of an individual be given by 


ULC LY: 


where consumption Č increases utility and supply of labour L decreases utility. Let 
the budget constraint of the individual be given by 


wL = C. 


Let the individual maximize utility with respect to consumption and the amount of 
labour supplied. 


(a) What is the optimal labour supply function (in implicit form)? How much does 
an individual consume? What is the indirect utility function? 


(b) Under what conditions does an individual increase labour supply when wages 
rise (no analytical solution required)? 


(c) Assume higher wages lead to increased labour supply. Does disutility arising 
from increased labour supply compensate utility from higher consumption? 
Does utility rise if there is no disutility from working? Start from the indirect 
utility function derived in a) and apply the proof of the envelope theorem and 
the envelope theorem itself. 


2. The envelope theorem II 
(a) Compute the derivative of the Bellman equation (3.4.6) without using the 


envelope theorem. Hint: Compute the derivative with respect to the state 
variable and then insert the first-order condition. 


(b) Do the same with (3.4.15) 
3. The additively separable objective function 


(a) Show that the objective function can be written as in (3.3.3). 
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(b) Find out whether (3.3.3) implies the objective function. (It does not.) 
4. Intertemporal and dynamic budget constraints 


(a) Show that the intertemporal budget constraint 


fg - sa d , 
Dia (ni) e- =a + J (i) tr (3.9.1) 
implies the dynamic budget constraint 
Qt+1 = (ay + Ut = er) (1 + rt) ; (3.9.2) 


(b) Under which conditions does the dynamic budget constraint imply the in- 
tertemporal budget constraint? 


(c) Now consider a¢41 = (1 + r) a¢+wi— e+. What intertemporal budget constraint 
does it imply? 


5. The standard saving problem 
Consider the objective function from (3.1.1), U; = 5%,97 ‘u(c-), and maximize 
it by choosing a consumption path {c,} subject to the constraint (3.6.6), kıyı = 
(1+ r+) ki + wE — c. The result is given by (3.6.7). 


(a) Solve this problem by dynamic programming methods. 


(b) Solve this by using the Lagrange approach. Choose a multiplier A; for an infinite 
sequence of constraints. 


(c) Solve this by using the Lagrange approach and construct an intertemporal 
budget constraint beforehand. 


6. Aggregation of optimal consumption rules 
Consider the optimality condition u’ (c4) = 6 (1+ 7141) w’ (c1) in (3.6.7) and derive 
the aggregate version (3.6.8). Find the assumptions required for the utility function 
for these steps to be possible. 


7. A benevolent central planner 
You are the omniscient omnipotent benevolent central planner of an economy. You 
want to maximize a social welfare function 


U; = X28 u (Cr) 


for your economy by choosing a path of aggregate consumption levels {C,} subject 
to a resource constraint 


Kipi T> Ky = Y (Kı, L) e ôK; =a Ci (3.9.3) 
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(a) Solve this problem by dynamic programming methods. 


(b) Discuss how the central planner result is related to the decentralized result 
from exercise 5. 


(c) What does the result look like for a utility function which is logarithmic and 
for one which has constant elasticity of substitution, 


Gry 4 
l-o 


u(C(T))=¢lnC (r) and u(C(r)) = ? (3.9.4) 


8. Environmental economics 
Imagine you are an economist only interested in maximizing the present value of 
your endowment. You own a renewable resource, for example a piece of forest. The 


amount of wood in your forest at a point in time t is given by x. Trees grow at 
b(x,) and you harvest at t the quantity c+. 


(a) What is the law of motion for x;? 


at is your objective function if prices at t per unit of wood is given by p, 
b) What i bjective function if pri tt it of d is gi b 
your horizon is infinity and you have perfect information? 


(c) How much should you harvest per period when the interest rate is constant? 
Does this change when the interest rate is time-variable? 


9. The 10k run - Formulating and solving a maximization problem 
You consider participation in a 10k run or a marathon. The event will take place 
in M months. You know that your fitness needs to be improved and that this will 
be costly: it requires effort ag which reduces utility u(.). At the same time, you 
enjoy being fast, i.e. utility increases the shorter your finish time /. The higher your 
effort, the shorter your finish time. 


(a) Formulate a maximization problem with 2 periods. Effort affects the finish time 
in M months. Specify a utility function and discuss a reasonable functional 
form which captures the link between finish time / and effort ao. 


(b) Solve this maximization problem by providing and discussing the first-order 
condition. 


10. A central planner 
Consider the objective function of a central planner, 


Uo = EX pu (C). (3.9.5) 
The constraint is given by 


Kipi = Y (Ki, Li) + (1 — ô) Ky am Ci. (3.9.6) 
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(a) Explain in words the meaning of the objective function and of the constraint. 


(b) Solve the maximization problem by first inserting (3.9.6) into (3.9.5) and then 
ul (Ct) _ | OY (Ki41,Lt41) = 
u’ (Ct+1) OK t41 


by optimally choosing K+. Show that the result is 
and discuss this in words. 


(c) Discuss why using the Lagrangian also requires maximizing with respect to K, 
even though K; is a state variable. 


Part II 


Deterministic models in continuous 
time 


TT 


Part II covers continuous time models under certainty. Chapter 4 first looks at dif- 
ferential equations as they are the basis of the description and solution of maximization 
problems in continuous time. First, some useful definitions and theorems are provided. 
Second, differential equations and differential equation systems are analyzed qualitatively 
by the so-called “phase-diagram analysis”. This simple method is extremely useful for 
understanding differential equations per se and also for later purposes for understand- 
ing qualitative properties of solutions to maximization problems and properties of whole 
economies. Linear differential equations and their economic applications are then finally 
analyzed before some words are spent on linear differential equation systems. 

Chapter 5 presents a new method for solving maximization problems - the Hamil- 
tonian. As we are now in continuous time, two-period models do not exist. A distinction 
will be drawn, however, between finite and infinite horizon models. In practice, this dis- 
tinction is not very important as, as we will see, optimality conditions are very similar for 
finite and infinite maximization problems. After an introductory example on maximiza- 
tion in continuous time by using the Hamiltonian, the simple link between Hamiltonians 
and the Lagrangian is shown. 

The solution to maximization problems in continuous time will consist of one or several 
differential equations. As a unique solution to differential equations requires boundary 
conditions, we will show how boundary conditions are related to the type of maximization 
problem analyzed. The boundary conditions differ significantly between finite and infinite 
horizon models. For the finite horizon models, there are initial or terminal conditions. 
For the infinite horizon models, we will get to know the transversality condition and other 
related conditions like the No-Ponzi-game condition. Many examples and a comparison 
between the present-value and the current-value Hamiltonian conclude this chapter. 

Chapter 6 solves the same kind of problems as chapter 5, but it uses the method 
of “dynamic programming”. The reason for doing this is to simplify understanding of 
dynamic programming in stochastic setups in Part IV. Various aspects specific to the use 
of dynamic programming in continuous time, e.g. the structure of the Bellman equation, 
can already be treated here under certainty. This chapter will also provide a comparison 
between the Hamiltonian and dynamic programming and look at a maximization problem 
with two state variables. An example from monetary economics on real and nominal 
interest rates concludes the chapter. 
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Chapter 4 


Differential equations 


There are many excellent textbooks on differential equations. This chapter will therefore 
be relatively short. Its objective is more to recap basic concepts taught in other courses 
and to serve as a background for later applications. 


4.1 Some definitions and theorems 


4.1.1 Definitions 
The following definitions are standard and follow Brock and Malliaris (1989). 


Definition 4.1.1 An ordinary differential equation system (ODE system) is of the type 


dx (t) 
dt 


where t lies between some starting point and infinity, t € |to, oo[ , x can be a vector, x € R” 
and f maps from R"*" into R”. When x is not a vector, i.e. for n =1, (4.1.1) obviously 
is an ordinary differential equation. 

An autonomous differential equation is an ordinary differential equation where f(.) is 
independent of time t, 


= i (t) = f (t,x (t)), (4.1.1) 


dx (t) 
dt 


The difference between a differential equation and a “normal” algebraic equation ob- 
viously lies in the fact that differential equations contain derivatives of variables like z (t). 
An example of a differential equation which is not an ODE is the partial differential 
equation. A linear example is 


aa a SPED p(t oD 


= i (t) = f (x(t). (4.1.2) 


where a(.), b(.), c(.) and p(.) are functions with “nice properties”. While in an ODE, 
there is one derivative (often with respect to time), a partial differential equation contains 
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derivatives with respect to several variables. Partial differential equations can describe 
e.g. a density and how it changes over time. Other types of differential equations include 
stochastic differential equations (see ch. 9), implicit differential equations (which are of 
the type g(«(t)) = f (t,x (t))), delay differential equations (t (t) = f (x(t — A))) and 
many other more. 


Definition 4.1.2 An initial value problem is described by 
p= f(t, z), £ (to) = Xo, tE lto, T] j 


where xo is the initial condition. 
A terminal value problem is of the form 


i= f (t,x), x (T) = tr, t€ lto, T], 


where xr is the terminal condition. 


4.1.2 Two theorems 


Theorem 4.1.1 Existence (Brock and Malliaris, 1989) 

If f (t,x) is a continuous function on rectangle L = {(t,x)| |t—to| < a, |x — zo| < b} 
then there exists a continuous differentiable solution x(t) on interval |t — to| < a that 
solves initial value problem 


t= f(t,x), x(to) = zo. (4.1.3) 


This theorem only proves that a solution exists. It is still possible that there are many 
solutions. 


Theorem 4.1.2 Uniqueness 
If f and Of /Ox are continuous functions on L, the initial value problem (4.1.3) has a 


unique solution for 
b 
te liyt oe 
f : a 


If this condition is met, an ODE with an initial or terminal condition has a unique 
solution. More generally speaking, a differential equation system consisting of n ODEs 
that satisfy these conditions (which are met in the economic problems we encounter here) 
has a unique solution provided that there are n boundary conditions. Knowing about a 
unique solution is useful as one knows that changes in parameters imply unambiguous 
predictions about changes in endogenous variables. If the government changes some tax, 
we can unambiguously predict whether employment goes up or down. 
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4.2 Analyzing ODEs through phase diagrams 


This section will present tools that allow us to determine properties of solutions of dif- 
ferential equations and differential equation systems. The analysis will be qualitative in 
this chapter as most economic systems are too non-linear to allow for an explicit general 
analytic solution. Explicit solutions for linear differential equations will be treated in 
ch. 4.3. 


4.2.1 One-dimensional systems 


We start with a one-dimensional differential equation « (t) = f (x (t)), where x € R and 
t > 0. This will also allow us to review the concepts of fixpoints, local and global stability 
and instability as used already when analyzing difference equations in ch. 2.5.4. 


e Unique fixpoint 
Let f (x) be represented by the graph in the following figure, with x (t) being shown 


on the horizontal axis. As f (x) gives the change of x (t) over time, t (t) is plotted on the 
vertical axis. 


Figure 4.2.1 Qualitative analysis of a differential equation 


As in the analysis of difference equations in ch. 2.5.4, we first look for the fixpoint of 
the underlying differential equation. A fixpoint is defined similarly in spirit but - thinking 
now in continuous time - differently in detail. 
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Definition 4.2.1 A fixpoint x* is a point where x(t) does not change. In continuous 
time, this means «(t) = 0 which, from the definition (4.1.2) of the differential equation, 
requires f (x*) = 0. 


The requirement that x (t) does not change is the similarity in spirit to the definition 
in discrete time. The requirement that f (x*) = 0 is the difference in detail: in discrete 
time as in (2.5.8) we required f (x*) = x*. Looking at the above graph of f (x), we find 
x* at the point where f (x) crosses the horizontal line. 

We then inquire the stability of the fixpoint. When z is to the left of x*, f(x) > 0 
and therefore x increases, t (t) > 0. This increase of x is represented in this figure by 
the arrows on the horizontal axis. Similarly, for x > x*, f (x) < 0 and x(t) decreases. 
We have therefore found that the fixpoint x* is globally stable and have also obtained a 
“feeling” for the behaviour of x (t) , given some initial conditions. 

We can now qualitatively plot the solutions with time t on the horizontal axis. As 
the discussion has just shown, the solution z (t) depends on the initial value from which 
we start, ie. on (0). For (0) > x*, x(t) decreases, for x(0) < 2*, x(t) increases: 
any changes over time are monotonic. There is one solution for each initial condition. 
The following figure shows three solutions of « (t) = f (x (t)), given three different initial 
conditions. 


x(t) 


Figure 4.2.2 Qualitative solutions of an ODE for three different initial conditions 


e Multiple fixpoints and equilibria 


Of course, more sophisticated functions than f(x) can be imagined. Now consider a 
differential equation « (t) = g (x (t)) where g(x) is non-monotonic as plotted in the next 
figure. 
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x(t) 


Figure 4.2.3 Multiple equilibria 


As this figure shows, there are four fixpoints. Looking at whether g (x) is positive or 
negative, we know whether x (t) increases or decreases over time. This allows us to plot 
arrows on the horizontal axis as in the previous example. The difference to before consists 
of the fact that some fixpoints are unstable and some are stable. Definition 4.2.1 showed 
us that the concept of a fixpoint in continuous time is slightly different from discrete 
time. However, the definitions of stability as they were introduced in discrete time can 
be directly applied here as well. Looking at xj, any small deviation of x from x} implies 
an increase or decrease of x. The fixpoint xj is therefore unstable, given the definition in 
ch. 2.5.4. Any small deviation x3, however, implies that x moves back to x3. Hence, x3 is 
(locally) stable. The fixpoint 23 is also unstable, while xj is again locally stable: While x 
converges to x} for any x > xj (in this sense x} could be called globally stable from the 
right), x converges to xj from the left only if x is not smaller than or equal to z3. 


Figure 4.2.4 Qualitative solutions of an ODE for different initial conditions IT 
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If an economy can be represented by such a differential equation « (t) = g (æ (t)), one 
would call fixpoints long-run equilibria. There are stable equilibria and unstable equilibria 
and it depends on the underlying system (the assumptions that implied the differential 
equation t = g (x)) which equilibrium would be considered to be the economically relevant 
one. 

As in the system with one fixpoint, we can qualitatively plot solutions of x (t) over time, 
given different initial values for x (0). This is shown in fig. 4.2.4 which again highlights 
the stability properties of fixpoints xj to x}. 


4.2.2 Two-dimensional systems I - An example 


We now extend our qualitative analysis of differential equations to two-dimensional sys- 
tems. This latter case allows for an analysis of more complex systems than simple one- 
dimensional differential equations. In almost all economic models with optimal saving 
decisions, a reduced form consisting of at least two differential equations will result. We 
start here with an example before we analyse two-dimensional systems more generally in 
the next chapter. 


e The system 
Consider the following differential equation system, 
tı = TĪ — T2, t =b + T] — x, 0<a<1l<b. 
Assume that for economic reasons we are interested in properties for x; > 0. 
e Fixpoint 


The first question is as always whether there is a fixpoint at all. In a two-dimensional 
system, a fixpoint x* = (a], x3) is two-dimensional as well. The fixpoint is defined such 
that both variables do not change over time, i.e. tı = t2 = 0. If such a point exists, it 
must satisfy 

ti = tz = 0 & (r4) = r3, z3 = b+ (r4). 


By inserting the second equation into the first, xt is determined by (xł)® = 6+(2%)7! and 
x, follows from z3 = (xt)°. Analyzing the properties of the equation (xł)® = b + (xt)7! 
would then show that xj is unique: The left-hand side increases monotonically from 0 to 
infinity for xj € [0, oo| while the right-hand side decreases monotonically from infinity to 
b. Hence, there must be an intersection point and there can be only one as functions are 
monotonic. As x7 is unique, so is 7} = (a7)°. 
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e Zero-motion lines and pairs of arrows 


Having derived the fixpoint, we now need to understand the behaviour of the system 
more generally. What happens to x; and x2 when (21,22) Æ (xj, x3)? To answer this 
question, the concept of zero-motion lines is very useful. A zero-motion line is a line for a 
variable x; which marks the points for which the variable x; does not change, i.e. t; = 0. 
For our two-dimensional differential equation system, we obtain two zero-motion lines, 


tı 20S T2 <T, 


In addition to the equality sign, we also analyse here at the same time for which values 
x; rises. Why this is useful will soon become clear. We can now plot the curves where 
ti = 0 in a diagram. In contrast to the one-dimensional graphs in the previous chapter, 
we now have the variables x; and zz on the axes (and not the change of one variable on 
the vertical axis). The intersection point of the two zero-motion lines gives the fix point 
x* = (xý, x3) which we derived analytically above. 


Figure 4.2.5 First steps towards a phase diagram 


In addition to showing where variables do not change, the zero-motion lines also delimit 
regions where variables do change. Looking at (4.2.1) again shows (why we used the > 
and not the = sign and) that the variable x, increases whenever x2 < xf. Similarly, the 
variable x2 increases, whenever £2 < b+ rI". The directions in which variables change 
can then be plotted into this diagram by using arrows. In this diagram, there is a pair of 
arrows per region as two directions (one for x1, one for £2) need to be indicated. This is in 
principle identical to the arrows we used in the analysis of the one-dimensional systems. 
If the system finds itself in one of these four regions, we know qualitatively, how variables 
change over time: Variables move to the south-east in region I, to the north-east in region 
II, to the north-west in region III and to the south-west in region IV. 
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e Trajectories 


Given the zero-motion lines, the fixpoint and the pairs of arrows, we are now able to 
draw trajectories into this phase diagram. We will do so and analyse the implications of 
pairs of arrows further once we have generalized the derivation of a phase diagram. 


4.2.3 Two-dimensional systems II - The general case 


After this specific example, we will now look at a more general differential equation system 
and will analyse it by using a phase diagram. 


e The system 


Consider two differential equations where functions f (.) and g(.) are continuous and 
differentiable, 
Ly = f (£1, £2) 5 To =g (£1, £2) : (4.2.2) 
For the following analysis, we will need four assumptions on partial derivatives; all of 
them are positive apart from fy (.), 


Toi (.) < 0, frz (.) > 0, Jeri (.) > 0, Gao (.) > 0. (4.2.3) 


Note that, provided we are willing to make the assumptions required by the theorems in 
ch. 4.1.2, we know that there is a unique solution to this differential equation system, i.e. 
x, (t) and x2 (t) are unambiguously determined given two boundary conditions. 


e Fixpoint 


The first question to be tackled is whether there is an equilibrium at all. Is there a 
fixpoint x* such that tı = tə = 0? To this end, set f (x1, £2) = 0 and g (£1, £2) = 0 and 
plot the implicitly defined functions in a graph. 


A 


Xz 


fx )=0 


g(x,,x,)=0 
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x, x 


Figure 4.2.6 Zero motion lines with a unique steady state 
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By the implicit function theorem - see (2.3.3) - and the assumptions made in (4.2.3), 
one zero motion line is increasing and one is decreasing. If we are further willing to assume 
that functions are not monotonically approaching an upper and lower bound, we know 
that there is a unique fixpoint (x7, 73) = 2°. 


e General evolution 


Now we ask again what happens if the state of the system differs from x*, i.e. if either 
zı Or £2 or both differ from their steady state values. To find an answer, we have to 
determine the sign of f (x1, £2) and g(#1,22) for some (z1, £2). Given (4.2.2), zı would 
increase for a positive f (.) and x2 would increase for a positive g(.). For any known 
functions f (v1, £2) and g (x1,£2) , one can simply plot a 3-dimensional figure with x, and 
£> on the axes in the plane and with time derivatives on the vertical axis. 


o 
— 
SS 


X 


Figure 4.2.7 A three-dimensional illustration of two differential equations and their zero- 
motion lines 


The white area in this figure is the horizontal plane, i.e. where tı and “2 are zero. 
The dark surface illustrates the law of motion for xı as does the grey surface for £z. The 
intersection of the dark surface with the horizontal plane gives the loci on which x; does 
not change. The same is true for the grey surface and x2. Clearly, at the intersection 
point of these zero-motion lines we find the steady state x*. 

When working with two-dimensional figures and without the aid of computers, we 
start from the zero-motion line for, say, xı and plot it into a “normal” figure. 
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T2 


Tı 
Figure 4.2.8 Step 1 in constructing a phase diagram 


Now consider a point (%1, 2). As we know that tı = 0 on f (z1, £2) = 0 and that 
from (4.2.3) fz, (.) > 0, we know that moving from Z, on the zero-motion line vertically 
to (#1, č2), f (-) is increasing. Hence, xı is increasing, tı > 0, at (Z1, %2). As this line of 
reasoning holds for any (1,2) above the zero-motion line, zı is increasing everywhere 
above f (x1, £2) = 0. As a consequence, zı is decreasing everywhere below the zero-motion 
line. This movement is indicated by the arrows in the above figure. 


T2 


Figure 4.2.9 Step 2 in constructing a phase diagram 


Let us now consider the second zero-motion line, #2 = 0 & g (z1, £2) = 0, and look 
again at the point (%,%2). When we start from the point č on the zero motion line 
and move towards (#1, 2), g (#1, £2) is decreasing, given the derivative g,; (.) > 0 from 
(4.2.3). Hence, for any points to the left of g (x1, £2) = 0, x2 is decreasing. Again, this is 
shown in the above figure. 
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e Fixpoints and trajectories 


We can now represent the directions in which zı and x2 are moving into a single 
phase-diagram by plotting one arrow each into each of the four regions limited by the 
zero-motion lines. Given that the arrows can either indicate an increase or a decrease for 
both x; and z2, there are two times two different combinations of arrows, i.e. four regions. 
When we add some representative trajectories, a complete phase diagram results. 


Figure 4.2.10 Phase diagram for a saddle point 


Adding trajectories is relatively easy when paying attention to the zero-motion lines. 
When trajectories are plotted “far away” from zero-motion lines, the arrow pairs indicate 
whether the movement is towards the “north-east”, the “south-east” or the other two 
possible directions. At point A in the phase diagram for a saddle point, the movement is 
towards the “north-west”. Note that the arrows represent first derivatives only. As second 
derivatives are not taken into account (usually), we do not know whether the trajectory 
moves more and more to the north or more and more to the west. The arrow-pairs are 
also consistent with wave-like trajectories, as long as the movement is always towards the 
north-west. 

Precise information on the shape of the trajectories is available when we look at the 
points where trajectories cross the zero-motion lines. On a zero-motion line, the variable 
to which this zero-motion line belongs does not change. Hence all trajectories cross 
zero-motion lines either vertically or horizontally. When we look at point B above, the 
trajectory moves from the north-west to the north-east region. The variable x, changes 
direction and begins to rise after having crossed its zero-motion line vertically. 

An example where a zero-motion line is crossed horizontally is point C. To the left of 
point C, x; rises and zə falls. On the zero-motion line for x2, £2 does not change but xı 
continues to rise. To the right of C, both zı and x2 increase. Similar changes in direction 
can be observed at other intersection points of trajectories with zero-motion lines. 
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4.2.4 Types of phase diagrams and fixpoints 
e Types of fixpoints 


As has become clear by now, the partial derivatives in (4.2.3) are crucial for the slope of 
the zero-motion lines and for the direction of movements of variables zı and x2. Depending 
on the signs of the partial derivatives, various phase diagrams can occur. As there are 
two possible directions for each variable, these phase diagrams can be classified into four 
typical groups, depending on the properties of their fixpoint. 


Definition 4.2.2 A fixpoint is called a 


center zero 

saddle point two . : . 
fous = all trajectories pass through the fixpoint 
node all 


and 5 at least one trajectory, both variables are non-monotonic 
all trajectories, one or both variables are monotonic 


A node and a focus can be either stable or unstable. 
e Illustration 


Here is now an overview of some typical phase diagrams. 


A Ral 


Figure 4.2.11 Phase diagram for a node 
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This first phase diagram shows a node. A node is a fixpoint through which all trajec- 
tories go and where the time paths implied by trajectories are monotonic for at least one 
variable. As drawn here, it is a stable node, i.e. for any initial conditions, the system ends 
up in the fixpoint. An unstable node is a fixpoint from which all trajectories start. A 
phase diagram for an unstable node would look like the one above but with all directions 
of motions reversed. 


X2 


Figure 4.2.12 Phase diagram for a focus 


A phase diagram with a focus looks similar to one with a node. The difference lies in 
the non-monotonic paths of the trajectories. As drawn here, x; or £2 first increase and 
then decrease on some trajectories. 


Xə x,=0 


Figure 4.2.13 Phase diagram for a center 
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A circle is a very special case for a differential equation system. It is rarely found 
in models with optimizing agents. The standard example is the predator-prey model, 
&=ax— Bry, y = —yy + Oxy, where a, 8, y, and 6 are positive constants. This is also 
called the Lotka-Volterra model. No closed-form solution has been found so far. 


e Limitations 


It should be noted that a phase diagram analysis allows us to identify a saddle point. 
If no saddle point can be identified, it is generally not possible to distinguish between a 
node, focus or center. In the linear case, more can be deduced from a graphical analysis. 
This is generally not necessary, however, as there is a closed-form solution. The definition 
of various types of fixpoints is then based on Eigenvalues of the system. See ch. 4.5. 


4.2.5 Multidimensional systems 


If we have higher-dimensional problems where x € R” and n > 2, phase diagrams are 
obviously difficult to draw. In the three-dimensional case, plotting zero motion surfaces 
sometimes helps to gain some intuition. A graphical solution will generally, however, not 
allow us to identify equilibrium properties like saddle-path or saddle-plane behaviour. 


4.3 Linear differential equations 


This section will focus on a special case of the general ODE defined in (4.1.1). The special 
aspect consists of making the function f (.) in (4.1.1) linear in x(t). By doing this, we 
obtain a linear differential equation, 


i(t) =a (t)z (t) + b(t), (4.3.1) 


where a (t) and b (t) are functions of time. This is the most general case for a one dimen- 
sional linear differential equation. 


4.3.1 Rules on derivatives 


Before analyzing (4.3.1) in more detail, we first need some other results which will be useful 
later. This section therefore first presents some rules on how to compute derivatives. It 
is by no means intended to be comprehensive or go in any depth. It presents rules which 
have shown by experience to be of importance. 


e Integrals 


Definition 4.3.1 A function F (x) = f f (x)dx is the indefinite integral of a function 
f (x) if 

d d 

EFE =E f f@)de=F@ (4.3.2) 
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This definition implies that there are infinitely many integrals of f (x). If F (x) is an 
integral, then F (x) +c, where c is a constant, is an integral as well. 


e Leibniz rule 
We present here a rule for computing the derivative of an integral function. Let there 


be a function z (x) with argument x, defined by the integral 


b(a) 


z(z)= - f (x,y) dy, 


where a(x), b (x) and f (x,y) are differentiable functions. Note that x is the only argu- 
ment of z, as y is integrated out on the right-hand side. Then, the Leibniz rule says that 
the derivative of this function with respect to x is 
d b(a) ð 
Tet a) febla- Fma EEE (433) 
£ a(x) £ 


The following figure illustrates this rule. 
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Figure 4.3.1 Illustration of the differentiation rule 


b(x) a 


Let x increase by a small amount. Then the integral changes at three margins: The 
upper bound, the lower bound and the function f (x,y) itself. As drawn here, the upper 
bound b (x) and the function f (x,y) increase and the lower bound a (x) decreases in x. As 
a consequence, the area below the function f (.) between bounds a (.) and b (.) increases 
because of three changes: the increase to the left because of a(.), the increase to the 
right because of b(.) and the increase upwards because of f (.) itself. Clearly, this figure 
changes when the derivatives of a(.), b(.) and f (.) with respect to x have a different sign 
than the ones assumed here. 
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e Another derivative with an integral 


Now consider a function of the type y = i f (x (i)) di. Functions of this type will be 
encountered frequently as objective functions, e.g. intertemporal utility or profit functions. 
What is the derivative of y with respect to x (i)? It is given by Oy/Ox (i) = f'(x (i)) . The 
integral is not part of this derivative as the derivative is computed for one specific x (i) 
and not for all z (i) with i lying between a and b. Note the analogy to maximizing a sum 
as e.g. in (3.1.4). The integration variable i here corresponds to the summation index T 
in (3.1.4). When one specific point 7 is chosen (say i = (a + b) /2), all derivatives of the 
other f (x (i)) with respect to this specific x (i) are zero. 


e Integration by parts (for indefinite and definite integrals) 


Proposition 4.3.1 For two differentiable functions u(x) and v (£), 


fe (x) v(x) dx = u(x) v(x) — fe (x) v (x) dz. (4.3.4) 
Proof. We start by observing that 
(u(x) v (z)) =w (z) v (z) +u (z) v (2), 


where we used the product rule. Integrating both sides by applying | - dx, gives 


u(x)v(x) = fe (a) v(e)da+ f ula (x) da. 


Rearranging gives (4.3.4). m 
Equivalently, one can show (see the exercise 8) that 


b b 
l rydt = [xy]? -f xydt. (4.3.5) 


4.3.2 Forward and backward solutions of a linear differential 
equation 


We now return to our linear differential equation « (t) = a (t) x (t) + b (t) from (4.3.1). It 
will now be solved. Generally speaking, a solution to a differential equation is a function 
x(t) which satisfies this equation. A solution can be called a time path of x when t 
represents time. 
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e General solution of the non-homogeneous equation 


The differential equation in (4.3.1) has, as all differential equations, an infinite number 
of solutions. The general solution reads 


a(t) = ef u(t)at (3 fe fe S a(t)dtp (t) it) : (4.3.6) 


Here, % is some arbitrary constant. As this constant is arbitrary, (4.3.6) indeed provides 
an infinite number of solutions to (4.3.1). 

To see that (4.3.6) is a solution to (4.3.1) indeed, remember the definition of what a 
solution is. A solution is a time path x (t) which satisfies (4.3.1). Hence, we simply need 
to insert the time path given by (4.3.6) into (4.3.1) and check whether (4.3.1) then holds. 
To this end, compute the time derivative of x(t), 


d 


Be tt) = e Oa (t) (3 + / e Jaap (t) it) + ef tate Sadh (t), 


where we have used the definition of the integral in (4.3.2), 4 f f (x) dx = f (x) . Note that 
we do not have to apply the product or chain rule since, again by (4.3.2), 4 fg(x)h(x)dz 
g(x) h (x). Inserting (4.3.6) gives z (t) = a(t) x (t) + b(t). Hence, (4.3.6) is a solution to 
(4.3.1). 

e Determining the constant 7 


To obtain one particular solution, some value x (to) at some point in time to has to be 
fixed. Depending on whether to lies in the future (where to is usually denoted by T) or in 
the past, t < T or tọ < t, the equation is solved forward or backward. 


backward forward 


to t T 


Figure 4.3.2 Illustrating backward solution (initial value problem) and forward solution 
(boundary value problem) 


We start with the backward solution, i.e. where tọ < t. Let the initial condition be 
x (to) = xo. Then the solution of (4.3.1) is 


= zoeo e / efr alulauy (7) dr. (4.3.7) 
to 
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Some intuition for this solution can be gained by considering special cases. Look first at 
the case where b (t) = 0 for all t (and therefore all 7). The variable x (t) then grows at a 
variable growth rate a (t) , t (t) /x (t) = a(t) from (4.3.1). The solution to this ODE is 


£ (t) = Jael a(r)dr = rect tol 


t 
where G = Leg baie is the average growth rate of a between to and t. The solution xe 
has the same structure as the solution for a constant a - this ODE implies an exponential 
increase of x (t). Looking at @ then shows that this exponential increase now takes place 
at the average of a(t) over the period we are looking at. 

Now allow for a positive b(t). The solution (4.3.7) says that when a b(7) is added 
in 7, the effect on x(t) is given by the initial b(7) times an exponential increase factor 
ef- «udu that takes the increase from 7 to t into account. As a b (T) is added at each 7, 
the outer integral jie .dr “sums” over all these individual contributions. 

The forward solution is required if tọ, which we rename T for ease of distinction, lies 
in the future of t, T > t. With a terminal condition x (T) = xr, the solution then reads 


alt—to] 


T 
x(t) = wpe Se adr _ / en Fe Ob (7) dr. (4.3.8) 


t 


A similar intuitive explanation as after (4.3.7) can be given for this equation. 

e Verification 

We now show that (4.3.7) and (4.3.8) are indeed a solution for (4.3.1). Using the 
Leibniz rule from (4.3.3), the time derivative of (4.3.7) is given by 

t t t 
& (t) = elo (4a (t) aq + eft 44H g / efr (odua (t) b (7) dr. 
to 

When we pull out a (t) and reinsert (4.3.7), we find 


t t t 
é(t) S(t) (ch Aner ey + f er e(dup (7) ir) + d(t) 


=a(t)x(t)+b(). 


This shows us that our function z (t) in (4.3.7) is in fact a solution of (4.3.1) as x(t) in 
(4.3.7) satisfies (4.3.1). 
The time derivative for the forward solution in (4.3.8) is 


T 
Hoce r OS / e- alway (7) dra (t) 


t 
T 
=a (t) G fy dTe -f eT Je udug (7) ir) + b(E) 
t 


=a(t)a(t)+b(f). 


Here, (4.3.8) was also reinserted into the second step. This shows that (4.3.8) is also a 
solution of (4.3.1). 
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4.3.3 Differential equations as integral equations 


Any differential equation can be written as an integral equation. While we will work 
with the “usual” differential equation representation most of the time, we introduce the 
integral representation here as it will be used frequently later when computing moments 
in stochastic setups. Understanding the integral version of differential equations in this 
deterministic setup allows for an easier understanding of integral representations of sto- 
chastic differential equations later. 


e The principle 
The non-autonomous differential equation « = f (t,x) can be written equivalently 
as an integral equation. To this end, write this equation as dx = f (t,x) dt or, after 


substituting s for t, as dx = f (s,x)ds. Now apply the integral fron both sides. This 
gives the integral version of the differential equation « = f (t,x) which reads 


/ dx = x(t) — x (0) -f f (s, x) ds. 
e An example 


The differential equation ¢ = a (t) x is equivalent to 


z (t) = To +f a (s) x (s) ds. (4.3.9) 


Computing the derivative of this equation with respect to time t gives, using (4.3.3), 
t (t) =a (t) x (t) again. 

The presence of an integral in (4.3.9) should not lead one to confuse (4.3.9) with 
a solution of « = a(t) in the sense of the last section. Such a solution would read 
a (t) = roef a2)4s, 


4.4 Examples 


4.4.1 Backward solution: A growth model 


Consider an example inspired by growth theory. Let the capital stock of the economy 
follow K = I — 6K, gross investment minus depreciation gives the net increase of the 
capital stock. Let gross investment be determined by a constant saving rate times output, 
I = sY (K) and the technology be given by a linear AK specification. The complete 
differential equation then reads 


K = sAK — ôK =(sA—6)K. 
Its solution is (see ch. 4.3.2) K (t) = ye4~%!. As in the qualitative analysis above, 
we found a multitude of solutions, depending on the constant y. If we specify an initial 


condition, say we know the capital stock at t = 0, i.e. K (0) = Ko, then we can fix the 
constant y by y = Ko and our solution finally reads K (t) = Koel- 
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4.4.2 Forward solution: Budget constraints 


As an application of differential equations, consider the budget constraint of a household. 
As in discrete time in ch. 3.5.2, budget constraints can be expressed in a dynamic and an 
intertemporal way. We first show here how to derive the dynamic version and then how 
to obtain the intertemporal version from solving the dynamic one. 


e Deriving a nominal dynamic budget constraint 


Following the idea of ch. 2.5.5, let us first derive the dynamic budget constraint. In 
contrast to ch. 2.5.5 in discrete time, we will see how straightforward a derivation is in 
continuous time. 

We start from the definition of nominal wealth. We have only one asset here. Nominal 
wealth is therefore given by a = kv, where k is the household’s physical capital stock and 
v is the value of one unit of the capital stock. One can alternatively think of k as the 
number of shares held by the household. By computing the time derivative, wealth of a 
household changes according to 

à = kv + ko. (4.4.1) 


If the household wants to save, it can buy capital goods. The household’s nominal savings 
in t are given by s = w*k+w—pe, the difference between factor rewards for capital (value 
marginal product times capital owned), labour income and expenditure. Dividing savings 
by the value of a capital good, i.e. the price of a share, gives the number of shares bought 
(or sold if savings are negative), 


: Kk a 
ae +w pe. 


- (4.4.2) 


Inserting this into (4.4.1), the equation for wealth accumulation gives, after reintroducing 
wealth a by replacing k by a/v, 


à = wk +w- pe+ kù = (w% +ò) k +w- pe = ~ Teti pe 
Defining the nominal interest rate as 
K . 
(22 (4.4.3) 
v 
we have the nominal budget constraint 
å = ia + w — pe. (4.4.4) 


This shows why it is wise to always derive a budget constraint. Without a derivation, 
(4.4.3) is missed out and the meaning of the interest rate 7 in the budget constraint is not 
known. 
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e Finding the intertemporal budget constraint 


We can now obtain the intertemporal budget constraint from solving the dynamic one 
in (4.4.4). Using the forward solution from (4.3.8), we take a (T) = ar as the terminal 
condition lying with T > t in the future. We are in t today. The solution is then 


T 
a(t) =e Si JAT ee f e7 Je ilu)du lw (T) — p (T)c(T)| dr & 


t 


[ P@wOar+a@)=DO)ar+ | D (rt) p(t) c(t) dr, 


where D (r) = e~ /¢ “4 defines the discount factor. As we have used the forward so- 
lution, we have obtained an expression which easily lends itself to an economic interpre- 
tation. Think of an individual who - at the end of his life - does not want to leave any 
bequest, i.e. ar = 0. Then, this intertemporal budget constraint requires that current 
wealth on the left-hand side, consisting of the present value of life-time labour income plus 
financial wealth a(t) needs to equal the present value of current and future expenditure 
on the right-hand side. 

Now imagine that ar > 0 as the individual does plan to leave a bequest. Then this 
bequest is visible as an expenditure on the right-hand side. Current wealth plus the 
present value of wage income on the left must then be high enough to provide for the 
present value D (T)ar of this bequest and the present value of consumption. 

What about debt in T? Imagine there is a fairy godmother who pays all debts left at 
the end of a life. With ar < 0 the household can consume more than current wealth a (t) 
and the present value of labour income - the difference is just the present value of debt, 
D (T)ar < 0. 

Now let the future point T in time go to infinity. Expressing limy_,,, a f (T)dr as 
Je f (7) dr, the budget constraint becomes 


[ Powder tae = jim Dare f Dp el)dr 


What would a negative present value of future wealth now mean, i.e. limr_... D (T)ar < 
0? If there was such a fairy godmother, having debt would allow the agent to permanently 
consume above its income levels and pay for this difference by accumulating debt. As fairy 
godmothers rarely exist in real life - especially when we think about economic aspects - 
economists usually assume that 


jm D(T)ar =0. (4.4.5) 
This condition is often called solvency or no-Ponzi game condition. Note that the no- 
Ponzi game condition is a different concept from the boundedness condition in ch. 5.3.2 
or the transversality condition in ch. 5.4.3. 
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e Real wealth 


We can also start from the definition of the household’s real wealth, measured in units 
of the consumption good, whose price is p. Real wealth is then a” = ©. The change in 
real wealth over time is then (apply the log on both sides and derive with respect to time), 


a kv p 
Inserting the increase into the capital stock obtained from (4.4.2) gives 


à wkk+w-—pe ù p (w*k+w-—pe)p o p 


a” vk v p ukp7! Dp 
Using the expression for real wealth a’, 

Fi Kk w Ù r p r m r r P r 

à =w f —c+-a — -a = a 4 —c+-a —-a 
P P v P v P v P 
K . . 

- (2 Fe Platte -cmra tho (4.4.6) 
v P P P 


Hence, the real interest rate r is - by definition - 


wE +ò ù 
r= — 


v p 


The difference to (4.4.3) simply lies in the inflation rate: Nominal interest rate minus in- 
flation rate gives real interest rate. Solving the differential equation (4.4.6) again provides 
the intertemporal budget constraint as in the nominal case above. 

Now assume that the price of the capital good equals the price of the consumption 
good, v = p. This is the case in an economy where there is one homogeneous output good 
as in (2.4.9) or in (9.3.4). Then, the real interest rate is equal to the marginal product of 
capital, r = w* /p. 


4.4.3 Forward solution again: capital markets and utility 


e The capital market no-arbitrage condition 


Imagine you own wealth of worth v (t). You can invest it on a bank account which 
pays a certain return r (t) per unit of time or you can buy shares of a firm which cost v (t) 
and which yield dividend payments z (t) and are subject to changes ù (t) in its worth. In 
a world of perfect information and assuming that in some equilibrium agents hold both 
assets, the two assets must yield identical income streams, 


r(t)u(t) =a (t) $0(t). (4.4.7) 
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This is a linear differential equation in v (t). As just motivated, it can be considered as 
a no-arbitrage condition. Note, however, that it is structurally equivalent to (4.4.3), i.e. 
this no-arbitrage condition can be seen to just define the interest rate r (t). 

Whatever the interpretation of this differential equation is, solving it forward with a 
terminal condition v (T) = ur gives according to (4.3.8) 


T 
v (t) =e FeO yp + J eh Me (7) dr 


t 


Letting T go to infinity, we have 


v (t) = if e7 hi udu (T)dr + jm ek EAT aa, 
t —>00 
This forward solution stresses the economic interpretation of v (t) : The value of an asset 
depends on the future income stream - dividend payments z (7) - that are generated from 
owning this asset. Note that it is usually assumed that there are no bubbles, i.e. the limit 
is zero so that the fundamental value of an asset is given by the first term. 

For a constant interest rate and dividend payments and no bubbles, the expression for 
v (t) simplifies to v = r/r. 


e The utility function 


Consider an intertemporal utility function as it is often used in continuous time models, 


U (t)= [ el™—4u (e(r)) dr. (4.4.8) 


This is the standard expression which corresponds to (2.2.12) or (3.1.1) in discrete time. 
Again, instantaneous utility is given by u (.). It depends here on consumption only, where 
households consume continuously at each instant 7. Impatience is captured as before by 
the time preference rate p: Higher values attached to present consumption are captured 
by the discount function e~’!"—4, whose discrete time analog in (3.1.1) is 87. 

Using (4.3.3), differentiating with respect to time gives us a linear differential equation, 


Ù (t) = —u (c (t)) + a £ [erly (c (r))] dr = —u(c(t)) + pU (t). 


This equation says that overall utility U (t) decreases, as time goes by, by instantaneous 
consumption u(c(t)). When t is over, the opportunity is gone: we can no longer enjoy 
utility from consumption at t. But U (t) also has an increasing component: as the future 
comes closer, we gain pU (t). 

Solving this linear differential equation forward gives 


U (t) =e lu (T) + ik elu (e(r)) dr. 


t 
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Letting T go to infinity, we have 


T 
U (t)= 1 e Plt (c(7))dr + jn er ay (T). 
t 


—0o 


The second term is related to the transversality condition. 


4.5 Linear differential equation systems 


A differential equation system consists of two or more differential equations which are 
mutually related to each other. Such a system can be written as 


i (t) = Aa (t) +b, 


where the vector x (t) is given by £ = (£1, %2, £3, Zn) , A is an n x n matrix with 
elements a;; and b is a vector b = (bj, bo, bs, ..., bn) . Note that elements of A and b can 
be functions of time but not functions of x. 

With constant coefficients, such a system can be solved in various ways, e.g. by deter- 
mining so-called Eigenvalues and Eigenvectors. These systems either result from economic 
models directly or are the outcome of a linearization of some non-linear system around 
a steady state. This latter approach plays an important role for local stability analyses 
(compared to the global analyses we undertook above with phase diagrams). These local 
stability analyses can be performed for systems of almost arbitrary dimension and are 
therefore more general and (for the local surrounding of a steady state) more informative 
than phase diagram analyses. 

Please see the references in “further reading” on many textbooks that treat these 
issues. 


4.6 Further reading and exercises 


There are many textbooks that treat differential equations and differential equation sys- 
tems. Any library search tool will provide many hits. This chapter owes insights to 
Gandolfo (1996) on phase diagram analysis and differential equations and - inter alia - 
to Brock and Malliaris (1989), Braun (1975) and Chiang (1984) on differential equations. 
See also Gandolfo (1996) on differential equation systems. The predator-prey model is 
treated in various biology textbooks. It can also be found on many sites on the Internet. 
The Leibniz rule was taken from Fichtenholz (1997) and can be found in many other 
textbooks on differentiation and integration. 
The AK specification of a technology was made popular by Rebelo (1991). 
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Exercises chapter 4 
Applied Intertemporal Optimization 


Using phase diagrams 


1. Phase diagram I 
Consider the following differential equation system, 


tı = f (z1, £2), t2 = g (£1, T2). 


Assume 
d d 
fr, (£1, £2) < 0, Ge (£1, £2) < 0, = <0, ae > 0. 

dx f(x1,22)=0 dx g(x1,%2)=0 

(a) Plot a phase diagram for the positive quadrant. 

(b) What type of fixpoint can be identified with this setup? 

2. Phase diagram II 
(a) Plot two phase diagrams for 
t=ry-a, y=y—ba>O0. (4.6.1) 


by varying the parameter b. 
(b) What type of fixpoints do you find? 


(c) Solve this system analytically. Note that y is linear and can easily be solved. 
Once this solution is plugged into the differential equation for x, this becomes 
a linear differential equation as well. 


3. Phase diagram II 


(a) Plot paths through points marked by a dot “-” in the figure below. 
(b) What type of fixpoints are A and B? 
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a Ea 
y y=0 


a x=0 


4. Local stability analysis 
Study local stability properties of the fixpoint of the differential equation system 
(4.6.1). 


5. Phase diagram and fixpoint 
Grossman and Helpman (1991) present a growth model with an increasing number of 
varieties. The reduced form of this economy can be described by a two-dimensional 
differential equation system, 


L a . =, l-a 
a wey ù (t) = pv (t) nly 


where 0 < a < 1 anda > 0. Variables v(t) and n (t) denote the value of the 
representative firm and the number of firms, respectively. The positive constants p 
and L denote the time preference rate and fix labour supply. 


(a) Draw a phase diagram (for positive n (t) and v (t)) and determine the fixpoint. 


(b) What type of fixpoint do you find? 


6. Solving linear differential equations 
Solve a y(t) + By (t) = 7, y (s) = 17 for 


(a) t> 5s, 


(b) t<s. 


(c) What is the forward and what is the backward solution? How do they relate 
to each other? 


4.6. 


T. 


10. 
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Comparing forward and backward solutions 
Remember that He f(z)dz = — ie f (z) dz for any well-defined z1, z2 and f(z). 
Replace T by to in (4.3.8) and show that the solution is identical to the one in 
(4.3.7). Explain why this must be the case. 


. Derivatives of integrals 


Compute the following derivatives. 


SETH 

(b) ale f(s, 7 ds, 

(c) z 4 f° f (u) dy, 

(d) Jf) 

(e) Show that the integration by parts formula I tydt = [xy]? — f? xydt holds. 


. Intertemporal and dynamic budget constraints 


Consider the intertemporal budget constraint which equates the discounted expen- 
diture stream to asset holdings plus a discounted income stream, 


f OEA f DIr, (4.6.2) 


where we defined 


D, (T) = exp |- E (s) is ; (4.6.3) 
A dynamic budget constraint reads 
EO+HÅ=rHA ++I). (4.6.4) 


(a) Show that solving the dynamic budget constraint Fa - intertemporal bud- 
get constraint if and only if limy_,.. A (T) exp |- fr ) dr =0. 


(b) Show that differentiating the intertemporal budget constraint yields the dy- 
namic budget constraint. 


A budget constraint with many assets 

Consider an economy with two assets whose prices are v; (t). A household owns 
n; (t) assets of each type such that total wealth at time t of the household is given 
by a (t) = v1 (t) nı (t) + v2 (t) no (t). Each asset pays a flow of dividends 7; (t). Let 
the household earn wage income w (t) and spend p (t) c(t) on consumption per unit 
of time. Show that the household’s budget constraint is given by 


a(t) =r (t)a(t) + w (t) = p Œ) c(t) 
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where the interest rates are defined by 


Ti (t) + 0; (t) 
Ui (t) 


and 6 (t) = v: (t) nı (t) /a (t) is defined as the share of wealth held in asset 1. 


r(t)=0(t)r(t)+1-6(t)ra(t), ni(t= 


11. Optimal saving 
Let optimal saving and consumption behaviour (see ch. 5, e.g. eq. (5.1.6)) be de- 
scribed by the two-dimensional system 


C=gc, a=rTra+w-c, 


where g is the growth rate of consumption, given e.g. by g = r—porg = (r — p) /o. 
Solve this system for time paths of consumption c and wealth a. 


12. ODE systems 
Study transitional dynamics in a two-country world. 


(a) Compute time paths for the number n’ (t) of firms in country i. The laws of 
motion are given by (Grossman and Helpman, 1991; Wälde, 1996) 


i’ = (nf +n?) Li-n'a(L+p), i=A,B, L=IA+L?, a,p>0. 


Hint: Eigenvalues are g = (1 — a) L — ap > 0 and A= ~a (L + p). 
(b) Plot the time path of n4. Choose appropriate initial conditions. 


Chapter 5 


Finite and infinite horizon models 


One widely used approach to solve deterministic intertemporal optimization problems in 
continuous time consists of using the so-called Hamiltonian function. Given a certain 
maximization problem, this function can be adapted - just like a recipe - to yield a 
straightforward result. The first section will provide an introductory example with a 
finite horizon. It shows how easy it can sometimes be to solve a maximization problem. 

It is useful to understand, however, where the Hamiltonian comes from. A list of 
examples can never be complete, so it helps to be able to derive the appropriate optimality 
conditions in general. This will be done in the subsequent section. Section 5.4 then 
discusses what boundary conditions for maximization problems look like and how they 
can be motivated. The infinite planning horizon problem is then presented and solved in 
section 5.3 which includes a section on transversality and boundedness conditions. Various 
examples follow in section 5.5. Section 5.7 finally shows how to work with present-value 
Hamiltonians and how they relate to current-value Hamiltonians (which are the ones used 
in all previous sections). 


5.1 Intertemporal utility maximization - an intro- 
ductory example 


5.1.1 The setup 


Consider an individual that wants to maximize a utility function similar to the one en- 
countered already in (4.4.8), 


Ug). = [ eP- In e(r) dr. (5.1.1) 


The planning period starts in t and stops in T > t. The instantaneous utility function is 
logarithmic and given by Inc(r). The time preference rate is p. The budget constraint 
of this individual equates changes in wealth, å (rT), to current savings, i.e. the difference 
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between capital and labour income, r (T) a (7)+w (7) , and consumption expenditure c (T), 
a(r) =r (r)a (7) +w(r) —e(7). (5.1.2) 


The maximization task consists of maximizing U (t) subject to this constraint by choosing 
a path of control variables, here consumption and denoted by {c(T)}. 


5.1.2 Solving by optimal control 


This maximization problem can be solved by using the present-value or the current-value 
Hamiltonian. We will work with the current-value Hamiltonian here and in what follows. 
Section 5.7 presents the present-value Hamiltonian and shows how it differs from the 
current-value Hamiltonian. The current-value Hamiltonian reads 


H =Inc(r) + A(t) [r (7) a (7) + w (7) -—c(7)], (5.1.3) 


where A (T) is a multiplier of the constraint. It is called the costate variable as it corre- 
sponds to the state variable a(r). In maximization problems with more than one state 
variable, there is one costate variable for each state variable. The costate variable could 
also be called Hamilton multiplier - similar to the Lagrange multiplier. We show further 
below that A (7) is the shadow price of wealth. The meaning of the terms state, costate 
and control variables is the same as in discrete time setups. 

Omitting time arguments, optimality conditions are 


—-—r=0, (5.1.4) 
\ = pA\—- — = p-r. (5.1.5) 
a 


The first-order condition in (5.1.4) is a usual optimality condition: the derivative of the 
Hamiltonian (5.1.3) with respect to the consumption level c must be zero. The second 
optimality condition - at this stage - just comes “out of the blue”. Its origin will be 
discussed in a second. Applying logs to the first first-order condition, — Inc = In À, and 
computing derivatives with respect to time yields —¢/c = \/X. Inserting into (5.1.5) gives 
the Euler equation 


ee ee ee ee (5.1.6) 
C C 


As this type of consumption problem was first solved by Ramsey in 1928 with some 
support by Keynes, a consumption rule of this type is often called Keynes-Ramsey rule. 

This rule is one of the best-known and most widely used in Economics. It says that 
consumption increases when the interest rate is higher than the time preference rate. One 
reason is that a higher interest rate implies - at unchanged consumption levels - a quicker 
increase in wealth. This is visible directly from the budget constraint (5.1.2). A quicker 
increase in wealth allows for a quicker increase in consumption. The second reason is that 
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a higher interest rate can lead to a change in the consumption level (as opposed to its 
growth rate). This channel will be analyzed in detail towards the end of ch. 5.6.1. 

Equations (5.1.2) and (5.1.6) form a two-dimensional differential equation system in 
a and c. This system can be solved given two boundary conditions. How these conditions 
can be found will be treated in ch. 5.4. 


5.2 Deriving laws of motion 


This subsection shows where the Hamiltonian comes from. More precisely, it shows how 
the Hamiltonian can be deduced from the optimality conditions resulting from a Lagrange 
approach. The Hamiltonian can therefore be seen as a shortcut which is quicker than the 
Lagrange approach but leads to (it needs to lead to) identical results. 


5.2.1 The setup 


Consider the objective function 


U (t) = / ely (y (Ps ee er hae (5.2.1) 


t 


which we now maximize subject to the constraint 


¥(7T) =Q(y (rT), 2(7), 7). (5.2.2) 


The function Q (.) is left fairly unspecified. It could be a budget constraint of a household, 
a resource constraint of an economy or some other constraint. We assume that Q (.) 
has “nice properties”, i.e. it is continuous and differentiable everywhere. The objective 
function is maximized by an appropriate choice of the path {z(7)} of control variables. 


5.2.2 Solving by the Lagrangian 


This problem can be solved by using the Lagrangian 


c= fee tu(rydr+ f n) -idr 


The utility function u (.) and the constraint Q (.) are presented as u(r) and Q (T), re- 
spectively. This shortens notation compared to full expressions in (5.2.1) and (5.2.2). 
The intuition behind this Lagrangian is similar to the one behind the Lagrangian in the 
discrete time case in (3.7.3) in ch. 3.7, where we also looked at a setup with many con- 
straints. The first part is simply the objective function. The second part refers to the 
constraints. In the discrete-time case, each point in time had its own constraint with its 
own Lagrange multiplier. Here, the constraint (5.2.2) holds for a continuum of points T. 
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Hence, instead of the sum in the discrete case we now have an integral over the product 
of multipliers 7 (7) and constraints. 
This Lagrangian can be rewritten as follows, 


pe i evel, (7) + 9 (7) Q(r) dr — f n(r) g(r) dr 


= i ety (T) + 9 (7) Q (T) dr +f n(t) y(t) dr — [n (r) y (7), (5.2.3) 


where the last step integrated by parts and [n (r) y (E is the integral function of 
n(T)y (T) evaluated at T minus its level at t. 

Now assume that we could choose not only the control variable z (Tr), but also the 
state variable y (T) at each point in time. The intuition for this is the same as in discrete 
time in ch. 3.7.2. Hence, we maximize the Lagrangian (5.2.3) with respect to y and z at 
one particular 7 € |t, T], i.e. we compute the derivative with respect to z (T) and y (T). 

For the control variable z (T), we get a first-order condition 


ely, (T) +n (7) Q: (7) =0 & u, (T) + enr) Q. (7) = 0 
When we define 
A (T) = eln (7), (5.2.4) 
we find 
uz (T) + A(T) Q- (7) =0. (5.2.5) 


For the state variable y (7) , we obtain 
eritu, (7) +9 (7) Qy (7) +i (7) = 0 S 
—uy (T) — 9 (7) el” AQ, (r) = eg (r). (5.2.6) 
Differentiating (5.2.4) with respect to time 7 and resinserting (5.2.4) gives 
A (P) = petn (7) + et (7) = pd (7) + erii (r), 
Inserting (5.2.6) and (5.2.4), we obtain 
À (T) = pÀ (T) — uy (T) — A(T) Q, (T). (5.2.7) 


Equations (5.2.5) and (5.2.7) are the two optimality conditions that solve the above 
maximization problem jointly with the constraint (5.2.2). We have three equations which 
fix three variables: The first condition (5.2.5) determines the optimal level of the control 
variable z. As this optimality condition holds for each point in time 7, it fixes an entire 
path for z. The second optimality condition (5.2.7) fixes a time path for À. By letting the 
costate À follow an appropriate path, it makes sure, that the level of the state variable 
(which is not instantaneously adjustable as the maximization of the Lagrangian would 
suggest) is as if it had been optimally chosen at each instant. Finally, the constraint 
(5.2.2) fixes the time path for the state variable y. 
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5.2.3 Hamiltonians as a shortcut 


Let us now see how Hamiltonians can be justified. The optimal control problem continues 
to be the one in ch.5.2.1 . Define the Hamiltonian similar to (5.1.3), 


H=u(r)+A(7) Q(z). (5.2.8) 


In fact, this Hamiltonian shows the general structure of Hamiltonians. Take the instan- 
taneous utility level (or any other function behind the discount term in the objective 
function) and add the costate variable A multiplied by the right-hand side of the con- 
straint. Optimality conditions are then 


H, =0, (5.2.9) 
\= pà- Hy. (5.2.10) 


These conditions were already used in (5.1.4) and (5.1.5) in the introductory example 
in the previous chapter 5.1 and in (5.2.5) and (5.2.7): When the derivatives H, and H, in 
(5.2.9) and (5.2.10) are computed from (5.2.8), this yields equations (5.2.5) and (5.2.7). 
Hamiltonians are therefore just a shortcut that allow us to obtain results faster than in 
the case where Lagrangians are used. Note for later purposes that both \ and 7 have 
time 7 as an argument. 

There is an interpretation of the costate variable À which we simply state at this 
point (see ch. 6.2 for a formal derivation): The derivative of the objective function with 
respect to the state variable at t, evaluated on the optimal path, equals the value of 
the corresponding costate variable at t. Hence, just as in the static Lagrange case, the 
costate variable measures the change in utility as a result of a change in endowment (i.e. 
in the state variable). Expressing this formally, define the value function as V (y (t)) = 
max,,(,)} U (t), identical in spirit to the value function in dynamic programming as we 
got to know it in discrete time. The derivative of the value function, the shadow price 
V” (y(t)), is then the change in utility when behaving optimally resulting from a change 
in the state y (t). This derivative equals the costate variable, V’ (y (t)) = À. 


5.3 The infinite horizon 


5.3.1 Solving by optimal control 
e Setup 
In the infinite horizon case, the objective function has the same structure as before in 


e.g. (5.2.1) only that the finite time horizon T is replaced by an infinite time horizon oo. 
The constraints are unchanged and the maximization problem reads 


max f ely (y (Tete) er har, 
{ar} Jt 
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subject to 


y (t) = Ye- (5.3.1) 


We need to assume for this problem that the integral ‘Pa e/l™Iy (.) dr converges for 
all feasible paths of y (T) and z (T), otherwise the optimality criterion must be redefined. 
This boundedness condition is important only in this infinite horizon case. Consider the 
following figure for the finite horizon case. 


el Fu(y(7),2( 7), T) 


LIE 


t T 


Figure 5.3.1 Bounded objective function for a finite horizon 


If individuals have a finite horizon (planning starts at t and ends at T) and the utility 
function u(.) is continuous over the entire planning period (as drawn), the objective 
function (the shaded area) is finite and the boundedness problem disappears. (As is clear 
from the figure, the condition of a continuous u(.) could be relaxed.) Clearly, making 
such an assumption is not always innocuous and one should check, at least after having 
solved the maximization problem, whether the objective function indeed converges. This 
will be done in ch. 5.3.2. 


e Optimality conditions 


The current-value Hamiltonian as in (5.2.8) is defined by H = u(r) + A(T) Q(t). 
Optimality conditions are (5.2.9) and (5.2.10), i.e. 
OH 


oH _ 
ðz 


Hence, we have identical optimality conditions to the case of a finite horizon. 
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5.3.2 The boundedness condition 


When introducing an infinite horizon objective function, it was stressed right after (5.3.1) 
that objective functions must be finite for any feasible paths of the control variable. 
Otherwise, overall utility U (t) would be infinitely large and there would be no objective 
for optimizing - we are already infinitely happy! This “problem” of unlimited happiness 
is particularly severe in models where control variables grow with a constant rate, think 
e.g. of consumption in a model of growth. A pragmatic approach to checking whether 
growth is not too high is to first assume that it is not too high, then to maximize the 
utility function and afterwards check whether the initial assumption is satisfied. 

As an example, consider the utility function U (t) = f° e-?l™~4u (e(r)) dr. The in- 
stantaneous utility function u (c(7)) is characterized by constant elasticity of substitution 
as in (2.2.10), 

c(r) e S17 
u (e(t) = =, so =. (5.3.2) 
Lo 

Assume that consumption grows with a rate of g, where this growth rate results from util- 
ity maximization. Think of this g as representing e.g. the difference between the interest 
rate and the time preference rate, corrected by intertemporal elasticity of substitution, 
as will be found later e.g. in the Keynes-Ramsey rule (5.6.8), i.e. ¢/c = (r—p)/o = g. 
Consumption at 7 > t is then given by c (T) = c(t) e97". With this exponential growth 
of consumption, the utility function becomes 


= ee be 1 
U(t) = (1-0)! eo E elt eG—oslr—ade + =|. 
t P 


This integral is bounded if and only if the boundedness condition 
(—e)g=p<0 


holds. This can formally be seen by computing the integral explicitly and checking under 
which conditions it is finite. Intuitively, this condition makes sense: Instantaneous utility 
from consumption grows by a rate of (1 — a) g. Impatience implies that future utility is 
discounted by the rate p. Only if this time preference rate p is large enough, the overall 
expression within the integral, e~?!"~4 (C (r) — 1) , will fall in 7. 


5.4 Boundary conditions and sufficient conditions 


So far, maximization problems were presented without boundary conditions. Usually, 
however, boundary conditions are part of the maximization problem. Without boundary 
conditions, the resulting differential equation system (e.g. (5.1.2) and (5.1.6) from the 
introductory example in ch. 5.1) has an infinite number of solutions and the level of 
control and state variables is not pinned down. We will now consider three cases. All 
cases will later be illustrated in the phase diagram of section 5.5. 
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5.4.1 Free value of the state variable at the endpoint 


Many problems are of the form (5.2.1) and (5.2.2), where boundary values for the state 
variable are given by 
y(t)=y, y(T) free. (5.4.1) 


The first condition is the usual initial condition. The second condition allows the state 
variable to be freely chosen for the end of the planning horizon. 

We can use the current-value Hamiltonian (5.2.8) to obtain optimality conditions 
(5.2.9) and (5.2.10). In addition to the boundary condition y (t) = y from (5.4.1), we 
have (cf. Feichtinger and Hartl, 1986, p. 20), 


d(T) =0. (5.4.2) 


With this additional condition, we have two boundary conditions which allows us to solve 
our differential equation system (5.2.9) and (5.2.10). This yields a unique solution and 
an example for this will be discussed further below in ch. 5.5. 


5.4.2 Fixed value of the state variable at the endpoint 


Now consider (5.2.1) and (5.2.2) with one initial and one terminal condition, 


y(t)=%, y (T) = yr- (5.4.3) 


In order for this problem to make sense, we assume that a feasible solution exists. This 
“should” generally be the case, but it is not obvious: Consider again the introductory 
example in ch. 5.1. Let the endpoint condition be given by “the agent is very rich in T”, 
i.e. a(T) =“very large”. If a(T) is too large, even zero consumption at each point in 
time, c (T) = 0 Y7 € [t, T], would not allow wealth a to be as large as required by a (T). 
In this case, no feasible solution would exist. 

We assume, however, that a feasible solution exists. Optimality conditions are then 
identical to (5.2.9) and (5.2.10), plus initial and boundary values (5.4.3). Again, two 
differential equations with two boundary conditions gives level information about the 
optimal solution and not just information about changes. 

The difference between this approach and the previous one is that, now, y (T) = yr 
is exogenously given, i.e. part of the maximization problem. Before the corresponding 
A(T’) = 0 was endogenously determined as a necessary condition for optimality. 


5.4.3 The transversality condition 


The analysis of the maximization problem with an infinite horizon in ch. 5.3 also led to 
a system of two differential equations. One boundary condition is provided by the initial 
condition in (5.3.1) for the state variable. Hence, again, we need a second condition to 
pin down the initial level of the control variable, e.g. the initial consumption level. 
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In the finite horizon case, we had terminal conditions of the type K (T) = Kr or 
A(T) = 0 in (5.4.2) and (5.4.3). As no such terminal T is now available, these conditions 
need to be replaced by alternative specifications. It is useful to draw a distinction between 
abstract conditions and conditions which have a practical value in the sense that they can 
be used to explicitly compute the initial consumption level. A first practically useful 
condition is the no-Ponzi game condition (4.4.5) resulting from considerations concerning 
the budget constraint, 


$ 
jim y (T) exp -f r (r) dr! =0 
=> t 
where r = 0H/ðy. Note that this no-Ponzi game condition can be rewritten as 


Jim eTA (T)y (T) = Jim n(T)y (T) =0, (5.4.4) 
where the fact that \/A = r — p implies that Ae™% = Agen JE v(@)4r was used. The 
formulations in (5.4.4) of the no-Ponzi game condition is frequently encountered as a 
second boundary condition. We will use it later in the example of ch. 5.6.1. 

A second useful way to determine levels of variables is the existence of a long-run 
steady state. With a well-defined steady state, one generally analyses properties on the 
saddle path which leads to this steady state. On this saddle path, the level of variables is 
determined, at least graphically. Numerical solutions also exist and sometimes analytical 
closed-form solutions can be found. Often, an analysis of the steady state alone is suffi- 
cient. An example where levels of variables are determined when analysing transitional 
dynamics on the saddle-path leading to the steady state is the central planner problem 
studied in ch. 5.6.3. 

Concerning abstract conditions, a condition occasionally encountered is the transver- 
sality condition (TVC), 


lim {7 (t) [y (4) — y* OIE = 9, 


t—oo 
where y* (t) is the path of y(t) for an optimal choice of control variables. There is a 
considerable literature on the necessity and sufficiency of the TVC and no attempt is 
made here to cover it. Various references to the literature on the TVC are in section 5.8 
on “further reading”. 


5.4.4 Sufficient conditions 


So far, we have only presented conditions that are necessary for a maximum. We do 
not yet know, however, whether these conditions are also sufficient. Sufficiency can be 
important, however, as it can be easily recalled when thinking of a static maximization 
problem. Consider max, f (x) where f’ (x*) = 0 is necessary for an interior maximum. 
This is not sufficient as f’ (x* + €) could be positive for any € Æ 0. 

For our purposes, necessary conditions are sufficient if either (i) the functions u (.) and 
Q (.) in (5.2.1) and (5.2.2) are concave in y and z and if 7 (7) is positive for all 7, (ii) 
Q (.) is linear in y and z for any 7(t) or (iii) Q (.) is convex and 7 (T) is negative for all 7. 
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The concavity of the utility function u (.) and constraint Q (.) can easily be checked 
and obviously hold, for example, for standard logarithmic and CES utility functions (as 
in (2.2.10) or (3.9.4)) and for constraints containing a technology as in (5.6.12) below. 
The sign of the shadow price can be checked by looking at the first order conditions as 
e.g. (5.1.4) or (5.6.13) later. As we usually assume that utility increases in consumption, 
we see that - for a problem to make economically sense - the shadow price is positive. 
Linearity in condition (ii) is often fulfilled when the constraint is e.g. a budget constraint. 

See “further reading” on references with a more formal treatment of sufficient condi- 
tions. 


5.5 Illustrating boundary conditions 


Let us now consider an example from microeconomics. We consider a firm that operates 
under adjustment costs. This will bring us back to phase-diagram analysis, to an under- 
standing of the meaning of fixed and free values of state variables at the end points for a 
finite planning horizon T, and to the meaning of transversality conditions for the infinite 
horizon. 


5.5.1 A firm with adjustment costs 


The maximization problem we are now interested in is a firm that operates under adjust- 
ment costs. Capital can not be rented instantaneously on a spot market but its installation 
is costly. The crucial implication of this simple generalization of the standard theory of 
production implies that firms “all of a sudden” have an intertemporal and no longer a 
static optimization problem. As one consequence, factors of production are then no longer 
paid their value marginal product as in static firm problems. 


e The model 


A firm maximizes the present value Io of its future instantaneous profits z (t), 


T 
Io = i er (t) dt, (5.5.1) 
0 
subject to a capital accumulation constraint 
K (t) =I (t) — 6K (t). (5.5.2) 


Gross investment T (t) minus depreciation ôK (t) gives the net increase of the firm’s capital 
stock. Economic reasoning suggests that K (t) should be non-negative. Instantaneous 
profits are given by the difference between revenue and cost, 


r (t) = pF (K (t)) — (I(t). (5.5.3) 
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Revenue is given by pF (K (t)) where the production technology F (.) employs capital 
only. Output increases in capital input, F” (.) > 0, but to a decreasing extent, F” (.) < 0. 
The firm’s costs are given by the cost function ®(/(t)). As the firm owns capital, it 
does not need to pay any rental costs for capital. Costs that are captured by ®(.) are 
adjustment costs which include both the cost of buying and of installing capital. The 
initial capital stock is given by K (0) = Ko. The interest rate r is exogenous to the firm. 
Maximization takes place by choosing a path of investment {J (t)}. 

The maximization problem is presented slightly differently from previous chapters. 
We look at the firm from the perspective of a point in time zero and not - as before - 
from a point in time t. This is equivalent to saying that we normalize t to zero. Both 
types of objective functions are used in the literature. One with normalization of today 
to zero as in (5.5.1) and one with a planning horizon starting in t. Economically, this is 
of no major importance. The presentation with t representing today as normally used in 
this book is slightly more general and is more useful when dynamic programming is the 
method chosen for solving the maximization problem. We now use a problem starting in 
0 to show that no major differences in the solution techniques arise. 


e Solution 


This firm obviously has an intertemporal problem, in contrast to the firms we encoun- 
tered so far. Before solving this problem formally, let us ask where this intertemporal 
dimension comes from. By looking at the constraint (5.5.2), this becomes clear: Firms 
can no longer instantaneously rent capital on some spot market. The firm’s capital stock 
is now a state variable and can only change slowly as a function of (positive or nega- 
tive) investment and depreciation. As an investment decision today has an impact on the 
capital stock “tomorrow”, i.e. the decision today affects future capital levels, there is an 
intertemporal link between decisions and outcomes at differing points in time. 

As discussed after (5.2.8), the current-value Hamiltonian combines the function after 
the discount term in the objective function, here instantaneous profits m (t), with the 
constraint, here I (t) — ô K (t), and uses the costate variable A (t). This gives 


H =n (t)+2(t) [1 () — 0K ()] = pF (K (t)) — @ 7 ()) +A) LE @) — OK A: 


Following (5.2.9) and (5.2.10), optimality conditions are 


H; = —®' (I (t)) + A(t) = 0, (5.5.4) 
\(t) =rA— Hx (t) =rA— pF" (K (t)) +6 
= (r+ 8) à — pF" (K (t)), (5.5.5) 


The optimality condition for A in (5.5.5) shows that the value marginal product of capital, 
pF" (K (t)), still plays a role and it is still compared to the rental price r of capital (the 
latter being adjusted for the depreciation rate), but there is no longer an equality as in 
static models of the firm. We will return to this point in exercise 2 of ch. 6. 
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In a long-run situation where the shadow price is constant, we see that value marginal 
productivity of capital, pF’ (K (t)), equals the interest rate plus the depreciation rate 
corrected for the shadow price. If, by (5.5.4), adjustment costs are given by investment 
costs only and the price of an investment good is identical to the price of the output good, 
we are back to equalities known from static models. 

Optimality conditions can be presented in a simpler way (i.e. with fewer endogenous 
variables). First, solve the first optimality condition for the costate variable and compute 
the time derivative, 


À = &" (I (t)) T(t). 
Second, insert this into the second optimality condition (5.5.5) to find 
D” (I (t)) F(t) = (r +6) P (I(t) — pF’ (K (0). (5.5.6) 


This equation, together with the capital accumulation constraint (5.5.2), is a two-dimensional 
differential equation system that can be solved, given the initial condition Kp and one 
additional boundary condition. 


e An example 


Now assume adjustment costs are of the form 
®(1) =v |I + I’/2]. (5.5.7) 


The price to be paid per unit of capital is given by the constant v and costs of installation 
are given by I?/2. This quadratic term captures the idea that installation costs are low 
and do not increase quickly, i.e. underproportionally to the new capital stock, at low 
levels of 7 but increase overproptionally when J becomes large. Then, optimality requires 
(5.5.2) and, from inserting (5.5.7) into (5.5.6), 


i(t)=(r+d)(1+1 (8) - Ep (K (t)) (5.5.8) 


A phase diagram using (5.5.2) and (5.5.8) is plotted in the following figure. As one 
can see, we can unambiguously determine that dynamic properties are represented by a 
saddle-path system with a saddle point as defined in def. 4.2.2. 
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Figure 5.5.1 A firm with adjustment cost 


We now have to select one of this infinite number of paths that start at Ko. What 
is the correct investment level J/g? This depends on how we choose the second boundary 
condition. 


5.5.2 Free value at the end point 


One modelling opportunity consists of leaving the value of the state variable at the end 
point open as in ch. 5.4.1. The condition is then À (T) = 0 from (5.4.2) which in the 
context of our example requires 


® (I(T)) =0e0[1+1(T)] =0 6 I(T) =-1. 


from the first-order condition (5.5.4) and the example for ® chosen in (5.5.7). In words, the 
trajectory where the investment level is minus one at T is the optimal one. Economically 
speaking, this means that the firm will sell capital at T (and also for some time before T) 
as we will now see. 

Let us now see how this information helps us to identify the level of investment and 
capital, i.e. the corresponding trajectory in the phase diagram by looking at fig. 5.5.1. 
Look at the trajectory starting at point A first. This trajectory crosses the zero-motion 
line for capital after some time and eventually hits the horizontal axis where J = 0. Have 
we now found a trajectory which satisfies all optimality conditions? Not yet, as we do not 
know whether the time needed to go from A to C is exactly of length T. If we started at 
B and went to D, we would also end up at J = 0, also not knowing whether the length is 
T. Hence, in order to find the appropriate trajectory, a numerical solution is needed. Such 
a solution would then compute various trajectories as the ones starting at A and B and 
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compare the length required to reach the horizontal axis. When the trajectory requiring 
T to hit the horizontal axis is found, levels of investment and capital are identified. 

There is also hope that searching for the correct trajectory does not take too long. 
We know that starting on the saddle path which leads to the steady state actually never 
brings us to the steady state. Capital K (t) and investment J (t) approach the steady 
state asymptotically but never reach it. As an example, the path for investment would 
look qualitatively like the graph starting with 293 in fig. 4.2.2 which approaches «* from 
above but never reaches it. If we start very close to the saddle path, let’s say at B, it takes 
more time to go towards the steady state and then to return than on the trajectory that 
starts at A. Time to reach the horizontal line is infinity when we start on the saddle path 
(i.e. we never reach the horizontal line). As time falls, the lower the initial investment 
level, the correct path can easily be found. 


5.5.3 Fixed value at the end point 


Let us now consider the case where the end point requires a fixed capital stock Kr. The 
first aspect to be checked is whether Kr can be reached in the planning period of length 
T. Maybe Kr is simply too large. Can we see this in our equations? 

If we look at the investment equation (5.5.2) only, any Kr can be reached by setting 
the investment levels 7 (T) just high enough. When we look at period profits in (5.5.3), 
however, we see that there is an upper investment level above which profits become 
negative. If we want to rule out negative profits, investment levels are bounded from 
above at each point in time by a(t) > 0 and some values at the endpoint Kr are not 
feasible. The maximization problem would have no solution. 

If we now look at a more optimistic example and let Kr not be too high, then we can 
find the appropriate trajectory in a similar way as before where J (T) = 0. Consider the 
Kr drawn in the figure 5.5.1. When the initial investment level is at point F, the level Kr 
will be reached faster than on a trajectory that starts between F and E. As time spent 
between Ko and Kr is monotonically decreasing, the higher the initial consumption level, 
i.e. the further the trajectory is away from the steady state, the appropriate initial level 
can again be easily found by numerical analysis. 


5.5.4 Infinite horizon and transversality condition 


Let us finally consider the case where the planning horizon is infinity, i.e. we replace T by 
oo. Which boundary condition shall we use now? Given the discussion in ch. 5.4.3, one 
would first ask whether there is some intertemporal constraint. As this is not the case 
in this model (an example for this will be treated shortly in ch. 5.6.1), one can add an 
additional requirement to the model analyzed so far. One could require the firm to be in 
a steady state in the long run. This can be justified by the observation that most firms 
have a relatively constant size over time. The solution of an economic model of a firm 
should therefore be characterized by the feature that, in the absence of further shocks, 
the firm size should remain constant. 
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The only point where the firm is in a steady state is the intersection point of the 
zero-motion lines. The question therefore arises where to start at Ko if one wants to 
end up in the steady state. The answer is clearly to start on the saddle path. As 
we are in a deterministic world, a transition towards the steady state would take place 
and capital and investment approach their long-run values asymptotically. If there were 
any unanticipated shocks which push the firm off this saddle path, investment would 
instantaneously be adjusted such that the firm is back on the saddle path. 


5.6 Further examples 


This section presents further examples of intertemporal optimization problems which can 
be solved by employing the Hamiltonian. The examples show both how to compute 
optimality conditions and how to understand the predictions of the optimality conditions. 


5.6.1 Infinite horizon - optimal consumption paths 


Let us now look at an example with infinite horizon. We focus on the optimal behaviour 
of a consumer. The problem can be posed in at least two ways. In either case, one part 
of the problem is the intertemporal utility function 


u= I * elu (e (7)) dr. (5.6.1) 


t 


Due to the general instantaneous utility function u(c(7)), it is somewhat more general 
than e.g. (5.1.1). The second part of the maximization problem is a constraint limiting 
the total amount of consumption. Without such a constraint, maximizing (5.6.1) would 
be trivial (or meaningless): With u’ (c(T)) > 0 maximizing the objective simply means 
setting c (T) to infinity. The way this constraint is expressed determines the way in which 
the problem is solved most straightforwardly. 


e Solving by Lagrangian 


The constraint to (5.6.1) is given by a budget constraint. The first way in which this 
budget constraint can be expressed is, again, the intertemporal formulation, 


i SoC ene Re f DC (5.6.2) 


where E (T) = p(r)c(r) and D, (r) = e f "du. The maximization problem is then 
given by: maximize (5.6.1) by choosing a path {c(T)} subject to the budget constraint 
(5.6.2). 
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We build the Lagrangean with A as the time-independent Lagrange multiplier 


i [ elu (e(r)) dr 


=] [Dear at f Dwaal, 


Note that in contrast to section 5.2.2 where a continuum of constraints implied a con- 
tinuum of Lagrange multipliers (or, in an alternative interpretation, a time-dependent 
multiplier), there is only one constraint here. 

The optimality conditions are the constraint (5.6.2) and the partial derivative with 
respect to consumption c (T) at one specific point 7 in time, i.e. the first-order condition 
for the Lagrangian, 


Ler) =e Plu! (e(r)) — AD, (r) p(7) = 0 
S D, (r) t ep (7)* = du! (elr). (5.6.3) 


Note that this first-order condition represents an infinite number of first-order conditions: 
one for each point 7 in time between t and infinity. See ch. 4.3.1 for some background on 
how to compute a derivative in the presence of integrals. Applying logs to (5.6.3) yields 


f r (u)du — p |T —t] — ln p (T) = ln à — ln w (e (T)). 
t 
Differentiating with respect to time 7 gives the Keynes-Ramsey rule 
= eet =r(T)—- —— — p. (5.6.4) 


The Lagrange multiplier \ drops out as it is not a function of time. Note that ae ae 


is Arrow’s measure of absolute risk aversion which is a measure of the curvature of the 
utility function. In our setup of certainty, it is more meaningful, however, to think of the 
curvature of a utility function as a measure of the intertemporal elasticity of substitution. 
Even though we are in continuous time now, the intertemporal elasticity of substitution 
can be defined as in (2.2.9). As we are not in discrete time now, we need to replace the 
distance of 1 from t to the next period t + 1 by a period of length A. One could then 
go through the same steps as after (2.2.9) and obtain identical results for a CES and 
logarithmic instantaneous utility function (see ex. 4). 


With a logarithmic utility function, u (c (T)) = lne (T), u’ (c(T)) = TA and u” (c (7)) = 
ST and the Keynes-Ramsey rule becomes 
aa h (5.6.5) 
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e Employing the Hamiltonian 


In contrast to above, the utility function (5.6.1) here is maximized subject to the 
dynamic (or flow) budget constraint, 


a(t)=r(t)a(t)+w(t)-—p(t)c(t). (5.6.6) 
The solution is obtained by solving exercise 1. 
e The interest rate effect on consumption 


As an application for the methods we have got to know so far, imagine there is a 
discovery of a new technology in an economy. All of a sudden, computers or cell-phones 
or the Internet is available on a large scale. Imagine further that this implies an increase 
in the returns on investment (i.e. the interest rate r): For any Euro invested, more comes 
out than before the discovery of the new technology. What is the effect of this discovery 
on consumption? 'To ask this more precisely: what is the effect of a change in the interest 
rate on consumption? 

To answer this question, we study a maximization problem as the one just solved, i.e. 
the objective function is (5.6.1) and the constraint is (5.6.2). We simplify the maximiza- 
tion problem, however, by assuming a CRRA utility function u (c (T)) = (c (r) — 1) /(1-0), 
a constant interest rate and a price being equal to one (imagine the consumption good is 
the numéraire). This implies that the budget constraint reads 


ie ee (7) dr = a(t) + i: eW lI (7) dr (5.6.7) 


t t 


and from inserting the CES utility function into (5.6.4), the Keynes-Ramsey rule becomes 


c(T) o 


One effect, the growth effect, is straightforward from (5.6.8) or also from (5.6.4). 
A higher interest rate, ceteris paribus, increases the growth rate of consumption. The 
second effect, the effect on the level of consumption, is less obvious, however. In order to 
understand it, we undertake the following steps. 

First, we solve the linear differential equation in c(r) given by (5.6.8). Following 
ch. 4.3, we find 


aR (5.6.8) 


c(t) =ec(t)hee 09, (5.6.9) 
Consumption, starting today in t with a level of c(t), grows exponentially over time at 
the rate (r — p) /o to reach the level c (T) at some future T > t. 


In the second step, we insert this solution into the left-hand side of the budget con- 
straint (5.6.7) and find 


i; el -te (thee dr = c(t) f en (FHP) rag 
t t 
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The simplification stems from the fact that c(t), the initial consumption level, can be 
pulled out of the term We e-'l"-le(r) dr, representing the present value of current and 
future consumption expenditure. Please note that c (t) could be pulled out of the integral 
also in the case of a non-constant interest rate. Note also that we do not need to know 
what the level of c(t) is, it is enough to know that there is some c(t) in the solution 
(5.6.9), whatever its level. 

With a constant interest rate, the remaining integral can be solved explicitly. First 
note that — (r — =e) must be negative. Consumption growth would otherwise exceed 
the interest rate and a boundedness condition for the objective function similar to the one 
in ch. 5.3.2 would eventually be violated. (Note, however, that boundedness in ch. 5.3.2 
refers to the utility function, here we focus on the present value of consumption.) Hence, 
we assume r > = < (1—a)r < p. Therefore, for the present value of consumption 
expenditure we obtain 


and, inserted into the budget constraint, this yields a closed-form solution for consump- 


tion, 

c(t) = p= or fa (t) +f ety (7) ar}. (5.6.10) 
t 

For the special case of a logarithmic utility function, the fraction in front of the curly 

brackets simplifies to p (as ø = 1). 

After these two steps, we have two results, both visible in (5.6.10). One result shows 
that initial consumption c(t) is a fraction out of wealth of the household. Wealth needs 
to be understood in a more general sense than usual, however: It is financial wealth a (t) 
plus, what could be called human wealth (in an economic, i.e. material sense), the present 
value of labour income, T e'l™lw (r)dr. Going beyond t today and realizing that this 
analysis can be undertaken for any point in time, the relationship (5.6.10) of course holds 
on any point of an optimal consumption path. The second result is a relationship between 
the level of consumption and the interest rate, our original question. 

We now need to understand the derivative dc (t) /dr in order to further exploit (5.6.10). 
If we focus only on the term in front of the curly brackets, we find for the change in the 
level of consumption when the interest rate changes 


de(t) dees > x 
iy = == = Le Oo 2 1. 


The consumption level increases when the interest rate rises if ø is larger than one, i.e. 
if the intertemporal elasticity of substitution o~' is smaller than unity. This is probably 
the empirically more plausible case (compared to ø < 1) on the aggregate level. There is 


5.6. Further examples 125 


micro-evidence, however, where the intertemporal elasticity of substitution can be much 
larger than unity. This finding is summarized in the following figure. 


Inc(t) ae 
c2(t) k 
c(t) ooo 
t time 


Figure 5.6.1 The effect of the interest rate on consumption growth and consumption 
level for an intertemporal elasticity of substitution smaller than one, i.e. o > 1 


e The boundary condition for the infinite horizon 


The steps we just went through are also an illustration of how to use the no-Ponzi game 
condition as a condition to obtain level information in an infinite horizon problem. We 
therefore just saw an example for the discussion in ch. 5.4.3 on transversality conditions. 

Solving a maximization problem by Hamiltonian requires a dynamic budget constraint, 
i.e. a differential equation. The solution is a Keynes-Ramsey rule, also a differential 
equation. These two differential equations require two boundary conditions in order to 
obtain a unique solution. One boundary condition is the initial stock of wealth, the second 
boundary condition is the no-Ponzi game condition. 

We just saw how this second condition can indeed be used to obtain level information 
for the control and the state variable: The No-Ponzi game condition allows us to obtain 
an intertemporal budget constraint of the type we usually want to work through solving 
the dynamic budget constraint - see ch. 4.4.2 on “Finding the intertemporal budget con- 
straint”. (In the example we just looked at, we did not need to derive an intertemporal 
budget constraint as it was already given in (5.6.2).) Using the Keynes-Ramsey rule in the 
way we just did provides the initial consumption level c(t). Hence, by using a boundary 
condition for this infinite horizon problem, we were able to obtain level information in 
addition to information on optimal changes. 

Note that the principle used here is identical to the one used in the analysis of level 
effects in ch. 3.4.3 on optimal R&D effort. 
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5.6.2 Necessary conditions, solutions and state variables 


The previous example provides a good opportunity to provide a more in-depth explanation 
of some concepts that were introduced before. 

A distinction was drawn between a solution and a necessary condition in ch. 2.2.2 
when discussing (2.2.6). The starting point of our analysis here is the Keynes-Ramsey 
rule (5.6.8) which is a necessary condition for optimal behaviour. The solution obtained 
here is given in (5.6.10) and is the outcome of solving the differential equation (5.6.8) and 
using the intertemporal budget constraint. Looking at these expressions clearly shows that 
(5.6.8), the outcome of modifying necessary conditions, contains much less information 
than the solution in (5.6.10). The Keynes-Ramsey rule provides information about the 
change of the control variable “only” while the solution provides information about the 
level. 

The solution in (5.6.10) is a closed-form or closed-loop solution. A closed-loop solution 
is a solution where the control variable is expressed as a function of the state variable 
and time. In (5.6.10), the state variable is a(t) and the function of time t is the integral 
JE e`" -w (r)dr. Closed-loop solutions stand in contrast to open-loop solutions where 
the control variable is a function of time only. This distinction becomes meaningful only 
in a stochastic world. In a deterministic world, any closed-loop solution can be expressed 
as a function of time only by replacing the state-variable by the function of time which 
describes its path. When we solve the budget constraint starting at some a (tọ) with 
to < t, insert c (T) from (5.6.9) into this solution for a (t) and finally insert this solution 
for a (t) into (5.6.10), we would obtain an expression for the control c(t) as a function of 
time only. 

The solution in (5.6.10) is also very useful for further illustrating the question raised 
earlier in ch. 3.4.2 on “what is a state variable?”. Defining all variables which influence 
the solution for the control variable as state variable, we clearly see from (5.6.10) that 
a(t) and the entire path of w(t), ie. w(7) for t < T < & are state variables. As we 
are in a deterministic world, we can reduce the path of w (T) to its initial value in t plus 
some function of time. What this solution clearly shows is that a(t) is not the only state 
variable. From solving the maximization problem using the Hamiltonian as suggested 
after (5.6.6) or from comparing with the similar setup in the introductory example in 
ch. 5.1, it is sufficient from a practical perspective, however, to take only a (t) as explicit 
state variable into account. The Keynes-Ramsey rule in (5.1.6) was obtained using the 
shadow-price of wealth only - see (5.1.5) - but no shadow-price for the wage was required 
in ch. 5.1. 


5.6.3 Optimal growth - the central planner and capital accumu- 
lation 


e The setup 


This example studies the classic central planner problem: First, there is a social welfare 
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function like (5.1.1), expressed slightly more generally as 


ma S oity C(r)) dr. 
pax | l 7) 


The generalization consists of the infinite planning horizon and the general instantaneous 
utility function (felicity function) u(c(7)). We will specify it in the most common version, 


Chee 
l-o 


u(C) = (5.6.11) 
where the intertemporal elasticity of substitution is constant and given by —1/c. 

Second, there is a resource constraint that requires that net capital investment is given 
by the difference between output Y (K, L), depreciation 6k and consumption C, 


K (t) =Y (K (t),L) -ôK (t) - C (t). (5.6.12) 


This constraint is valid for t and for all future points in time. Assuming for simplicity that 
the labour force L is constant, this completely describes this central planner problem. 
The planner’s choice variable is the consumption level C (7T) , to be determined for each 
point in time between today t and the far future oo. The fundamental trade-off lies in the 
utility increasing effect of more consumption visible from (5.6.11) and the net-investment 
decreasing effect of more consumption visible from the resource constraint (5.6.12). As 
less capital implies less consumption possibilities in the future, the trade-off can also be 
described as lying in more consumption today vs. more consumption in the future. 


e The Keynes-Ramsey rule 


Let us solve this problem by employing the Hamiltonian consisting of instantaneous 
utility plus A(t) multiplied by the relevant part of the constraint, H = u(C(t)) + 
A (t) [Y (K (t), L)— ôK (t) — C (t)]. Optimality conditions are 


wu (C (t)) =A (¢), (5.6.13) 


H E 


- zp = PAL) -A O) [Ve (K 6), L) — 4]. 


Differentiating the first-order condition (5.6.13) with respect to time gives u” (C (t)) C (t) = 
À (t). Inserting this and (5.6.13) into the second condition again gives, after some rear- 
ranging, WiC) 
= aC)” = Yx(K (t), L)—ô— p. 
This is almost identical to the optimality rule we obtained on the individual level in 
(5.6.4). The only difference lies in aggregate consumption C instead of c and Yx (K (t), L)— 
ô instead of r (T) — aD., Instead of the real interest rate on the household level, we here 


have the marginal productivity of capital minus depreciation on the aggregate level. 
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(t)) = 


If we assumed a logarithmic instantaneous utility ee —u" (C (t)) /w’ (C 
es p, similar 
11), 


1/C (t) and the Keynes-Ramsey rule would be C (t) /C (t) = Yx (K (t), L)— 
o (5.1.6) or (5.6.5). In our case of the more general CES specification in (5.6 
u” (C (t)) /u’ (C (t)) = 0 /C (t) such that the Keynes-Ramsey rule reads 


we find 


C(t) Yx(K(t),L)-6-p 
a) = 5 . (5.6.14) 


This could be called the classic result on optimal consumption in general equilibrium. 
Consumption grows if marginal productivity of capital exceeds the sum of the depreciation 
rate and the time preference rate. The higher the intertemporal elasticity of substitution 
1/o, the stronger consumption growth reacts to the differences Yg (K (t), L) — ô — p. 


e A phase diagram analysis 


The resource constraint of the economy in (5.6.12) plus the Keynes-Ramsey rule in 
(5.6.14) represent a two-dimensional differential equation system which, given two bound- 
ary conditions, give a unique solution for time paths C (t) and K (t). These two equations 
can be analyzed in a phase diagram. This is probably the phase diagram taught most 
often in Economics. 


Figure 5.6.2 Optimal central planner consumption 


Zero-motion lines for capital and labour, respectively, are given by 


K (t) >0@ C(t) <Y (K (t), L) — 6K (t), (5.6.15) 
C(t) > 06 Yx(K (t),L) > +p, (5.6.16) 


when the inequality signs hold as equalities. Zero motion lines are plotted in the above 
figure. 

When consumption lies above the Y (K (t), L) — ôK (t) line, (5.6.15) tells us that 
capital decreases, below this line, capital increases. When the marginal productivity of 
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capital is larger than ô + p, i.e. when the capital stock is sufficiently small, (5.6.16) tells 
us that consumption increases. These laws of motion are also plotted in figure 5.6.2. This 
allows us to draw trajectories A, B and C which all satisfy (5.6.12) and (5.6.14). Hence, 
again, we have a multitude of solutions for a differential equation system. 

As always, boundary conditions allow us to “pick” the single solution to this system. 
One boundary condition is the initial value Ko of the capital stock. The capital stock is 
a state variable and therefore historically given at each point in time. It can not jump. 
The second boundary condition should fix the initial consumption level Co. Consumption 
is a control or choice variable and can therefore jump or adjust to put the economy on 
the optimal path. 

The condition which provides the second boundary condition is, formally speaking, 
the transversality condition (see ch. 5.4.3). Taking this formal route, one would have to 
prove that starting with Ko at the consumption level that puts the economy on path A 
would violate the TVC or some No-Ponzi game condition. Similarly, it would have to be 
shown that path C or any path other than B violates the TVC as well. Even though not 
often admitted, this is not often done in practice. (For an exception to the application 
of the No-Ponzi game condition, see ch. 5.6.1 or exercise 5 in ch. 5.) Whenever a saddle 
path is found in a phase diagram, it is argued that the saddle path is the equilibrium 
path and the initial consumption level is such that the economy finds itself on the path 
which approaches the steady state. While this is a practical approach, it is also formally 
satisfied as this path satisfies the TVC indeed. 


5.6.4 The matching approach to unemployment 


Matching functions are widely used in, for example, Labour economics and Monetary 
economics. Here we will present the background for their use in labour market modelling. 


e Aggregate unemployment 


The unemployment rate in an economy is governed by two factors: the speed with 
which new employment is created and the speed with which existing employment is de- 
stroyed. The number of new matches per unit of time dt is given by a matching function. 
It depends on the number of unemployed U, i.e. the number of those potentially available 
to fill a vacancy, and the number of vacancies V, m = m (U,V). The number of filled 
jobs that are destroyed per unit of time is given by the product of the separation rate 
s and employment, sL. Combining both components, the evolution of the number of 
unemployed over time is given by 


U = sL -m (U,V). (5.6.17) 


Defining employment as the difference between the size of the labour force N and the 
number of unemployed U, L = N — U, we obtain 


U = s|N —- U| -m (U, V) & ù = s [1 — u] — m (u, V/ N), 
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where we divided by the size of the labour force N in the last step to obtain the un- 
employment rate u = U/N. We also assumed constant returns to scale in the matching 
function m (.). Defining labour market tightness by 0 = V/U, the matching function can 
further be rewritten as 


aa en e a e eas 


and we obtain ù = s [1 — u] — 8q (0) u which is equation (1.3) in Pissarides (2000). 
Clearly, from (5.6.17), one can easily obtain the evolution of employment, again using 
the definition L = N — U, 


L=m(N-—L,V)-sL. (5.6.19) 
e Optimal behaviour of large firms 


Given this matching process, the choice variable of a firm 2 is no longer employment L; 
but the number of vacancies V; it creates. There is an obvious similarity to the adjustment 
cost setup in ch. 5.5.1. Here, the firm is no longer able to choose labour directly but only 
indirectly through vacancies. With adjustment costs, the capital stock is chosen only 
indirectly through investment. 

The firm’s objective is to maximize its present value, given by the integral over dis- 
counted future profits, o 

max | elt, (r)dr. 
{Vi Ki} Jt 


Profits are given by 


Each vacancy implies costs y measured in units of the output good Y. The firm rents 
capital K; from the capital market and pays interest r and a wage rate w per workers. 
The employment constraint is 

v, 

V 

The constraint now says - in contrast to (5.6.19) - that only a certain share of all matches 
goes to the firm under consideration and that this share is given by the share of the firm’s 
vacancies in total vacancies, V;/V. Alternatively, this says that the “probability” that 
a match in the economy as a whole fills a vacancy of firm 7 is given by the number of 
matches in the economy as a whole divided by the total number of vacancies. As under 
constant returns to scale for m and using the definition of q (0) implicitly in (5.6.18), 
m(N — L, V)/V =m(U/V,1) = q (0) , we can write the firm’s constraint as 


Li = q (0) V; — sLi. (5.6.20) 


Assuming small firms, this rate q (0) can safely be assumed to be exogenous for the firm’s 
maximization problem. 
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The current-value Hamiltonian for this problem reads H = 7; + Aby i.e. 
H =Y (K;, Li) — rK; — wl, — Vi + ài lg (0) Vi — sLil, 
and the first-order conditions for capital and vacancies are 
Ax, = Yr, (Ki, Li) -r =0, Ay, = —y + àq (0) =0. (5.6.21) 
The optimality condition for the shadow price of labour is 
X = rà; — Ay, = rài — (Yr, (Ki, Li) — w — àis) = (r + s$) ài — Yz, (Ka, Li) + w. 


The first condition for capital is the usual marginal productivity condition applied to 
capital input. With constant-returns-to-scale production functions, this condition fixes 
the capital to labour ratio for each firm. This implies that the marginal product of labour 
is a function of the interest only and therefore identical for all firms, independent of the 
labour stock, i.e. the size of the firm. This changes the third condition to 


\=(r+s)A—-Y,(K,L)+w (5.6.22) 


which means that the shadow prices are identical for all firms. 

The first-order condition for vacancies, written as y = q(@) A says that the marginal 
costs y of a vacancy (which in this special case equal average and unit costs) must be 
equal to revenue from a vacancy. This expected revenue is given by the share q (0) of 
vacancies that yield a match times the value of a match. The value of a match to the 
firm is given by A, the shadow price of labour. The link between À and the value of an 
additional unit of labour (or of the state variable, more generally speaking) is analyzed 
in ch. 6.2. 


e General equilibrium 


It appears as if the first-order condition for vacancies (5.6.21) was independent of the 
number of vacancies opened by the firm. In fact, given this structure, the individual firm 
follows a bang-bang policy. Either the optimal number of vacancies is zero or infinity. 
In general equilibrium, however, a higher number of vacancies increases the rate q (0) in 
(5.6.20) with which existing vacancies get filled (remember that 6 = V/U). Hence, this 
second condition holds in general equilibrium and we can compute by differentiating with 


respect to time 
A/A = —ġ (9) /q (0). 


The third condition (5.6.22) can then be written as 
q (0) 


Ta ae [Yn (K, L) — w] (5.6.23) 


which is an equation depending on the number of vacancies and employment only. This 
equation, together with (5.6.19), is a two-dimensional differential equation system which 
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determines V and L, provided there are 2 boundary conditions for V and L and wages and 
the interest rate are exogenously given. (In a more complete general equilibrium setup, 
wages would be determined e.g. by Nash-bargaining. For this, we need to derive value 
functions which is done in ch. 11.2). 


e An example for the matching function 
Assume the matching function has constant returns to scale and is of the CD type, 


m = U°V®. As q (0) =m(N—L,V) /V, it follows that in = = A Given our Cobb- 
Douglas assumption, we can write this as 


Hence, the vacancy equation (5.6.23) becomes 


Ù Vi q (0) 
“a + (1- BG =r s— 7 [Yr (K, L) — w] & 
j = B 
a= py arte COP i (kt) -uta lsr- 


where the last equality used q (6) = (N — L)“ V8"? and (5.6.17) with m = U°V®%. Again, 
this equation with (5.6.19) is a two-dimensional differential equation system which deter- 
mines V and L,as just described in the general case. 


5.7 The present value Hamiltonian 


5.7.1 Problems without (or with implicit) discounting 


e The problem and its solution 


Let the maximization problem be given by 


max f F (y (T), z (T), T)dT (5.7.1) 

subject to 
y(t) =Q(y (7), 2(7),7) (5.7.2) 
y(t) =y y(T) free (5.7.3) 


where y (7) € R”, z (T) € R” and Q = (Qı (y (T), z (T), T), 
path is a pair (y(7), z(7)) which satisfies (5.7.2) and (5.7.3 
variables, y(7) is the vector of state variables. 


i laa ae Or 7. A feasible 


(T) is the vector of control 
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Then define the (present-value) Hamiltonian H? as 
H? = F(r) + n(r) Q(z), (5.7.4) 
where 7(T) is the costate variable, 


n (T) = (m (T); n2 (T); +s Mn (7) (5.7.5) 


Necessary conditions for an optimal solution are 


H? = 0. (5.7.6) 
a(t) =—-H?, (5.7.7) 
and 
n(T)=0, 


(Kamien and Schwartz, p. 126) in addition to (5.7.2) and (5.7.3). In order to have a 
maximum, we need second order conditions H,, < 0 to hold (Kamien and Schwartz). 


e Understanding its structure 


The first m equations (5.7.6) (z (T) € R™) solve the control variables z(7) as a function 
of state and costate variables, y(7) and 7(r). Hence 


The next 2n equations (5.7.7) and (5.7.2) (y(t) € R”) constitute a 2n dimensional dif- 
ferential equation system. In order to solve it, one needs, in addition to the n initial 
conditions given exogenously for the state variables by (5.7.3), n further conditions for 
the costate variables. These are given by boundary value conditions 7 (T) = 0. 
Sufficiency of these conditions results from theorem as e.g. in section 5.4.4. 


5.7.2 Deriving laws of motion 


As in the section on the current-value Hamiltonian, a ”derivation” of the Hamiltonian 
starting from the Lagrange function can be given for the present value Hamiltonian as 
well. 


e The maximization problem 


Let the objective function be (5.7.1) that is to be maximized subject to the constraint 
(5.7.2) and, in addition, a static constraint 


G(y(7),2(7),7) = 0. (5.7.8) 
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This maximization problem can be solved by using a Lagrangian. The Lagrangian 
reads 


g=] F()+n(7)(Q(0) -—9(7)) —€(7) G(), t) dr 


-f F()+n(7)Q() EM G(Q)ar- | nt) y (7) dr 


Using the integration by parts rule T tydt = — i xydt + [xy]? from (4.3.5), integrating 
the last expression gives 


f n(rya(ryae=— f ñ (T) y (T) dr + [n (T) y (P); 
and the Lagrangian reads 
L=] PO +n QO +AU) -EGC dr 


“Jorio 


This is now maximized with respect to y (T) and z (T), both the control and the state 
variable. We then obtain conditions that are necessary for an optimum. 


Fy (.) + (7) Q:z (.) — € (7) Gz (.) = 0 (5.7.9) 
Fy (.) +7 (7) Qy (-) +9 (7) — € (7) Gy (.) = 0 (5.7.10) 

The last first-order condition can be rearranged to 
(7) = —1 (7) Qy (.) — Fy (-) + € (7) Gy (-) (5.7.11) 


These necessary conditions will be the ones used regularly in maximization problems. 
e The shortcut 


As it is cumbersome to start from a Lagrangian for each dynamic maximization prob- 
lem, one can define the Hamiltonian as a shortcut as 


HP =F(.)+n(r) QO) -E(7) G(). (5.7.12) 

Optimality conditions are then 
H? =0, (5.7.13) 
ù (r) =-HF. (5.7.14) 


which are the same as (5.7.9) and (5.7.11) above and (5.7.6) and (5.7.7) in the last section. 
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5.7.3 The link between CV and PV 


If we solve the problem (5.2.1) and (5.2.2) via the present value Hamiltonian, we would 
start from 


H” (7) =e IG (.) +n (7) Q(.), (5.7.15) 
first-order conditions would be 
4 (LT) =0, (5.7.16) 
P 
a = eG, (.) +Q: (.) = 0, (5.7.17) 
: OH? PAS 
ù (T) = -= = -eG (.) — nQy (.) (5.7.18) 


and we would be done with solving this problem. 
e Simplification 


We can simplify the presentation of first-order conditions, however, by rewriting 
(5.7.17) as 


Gz (7) + A(7T)Qz(T) =0 (5.7.19) 
where we used the same definition as in (5.2.4), 
A (T) = e7 (r). (5.7.20) 


Note that the argument of the costate variables is always time 7 (and not time t). 
When we use this definition in (5.7.18), this first-order condition reads 


erli (r) = -Gy () — AQy (). 
Replacing the left-hand side by 
eth (7) =N =p rS A=): 
which follows from computing the time 7 derivative of the definition (5.7.20), we get 
A= pA— Gy ANTIQ: (5.7.21) 
Inserting the definition (5.7.20) into the Hamiltonian (5.7.15) gives 
ell" HP (t) = G(.) +AQ (.) <=> H(t) = G(.) +AQ(.) 
which defines the link between the present value and the current-value Hamiltonian as 
H" (t) =e?" 4H? (t) 
e Summary 


Hence, instead of first-order conditions (5.7.17) and (5.7.18), we get (5.7.19) and 
(5.7.21). As just shown these first-order conditions are equivalent. 
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5.8 Further reading and exercises 


The classic reference for the optimal consumption behaviour of an individual household 
analyzed in ch. 5.6.1 is Ramsey (1928). The paper credits part of the intuitive explanation 
to Keynes - which led to the name Keynes-Ramsey rule. See Arrow and Kurz (1969) for 
a detailed analysis. 

There are many texts that treat Hamiltonians as a maximization device. Some ex- 
amples include Dixit (1990, p. 148), Intriligator (1971), Kamien and Schwartz (1991), 
Leonard and Long (1992) or, in German, Feichtinger and Hartl (1986). Intriligator pro- 
vides a nice discussion of the distinction between closed- and open-loop controls in his 
ch. 11.3. The corresponding game-theory definitions for closed-loop and open-loop strate- 
gies (which are in perfect analogy) are in Fudenberg and Tirole (1991). 

On sufficient conditions, see Kamien and Schwartz (1991, ch. 3 and ch. 15). 

The literature on transversality conditions includes Mangasarian (1966), Arrow (1968), 
Arrow and Kurz (1970), Araujo and Scheinkman (1983), Léonard and Long (1992, p. 288- 
289), Chiang (1992, p. 217, p. 252) and Kamihigashi (2001). Counterexamples that the 
TVC is not necessary are provided by Michel (1982) and Shell (1969). See Buiter and 
Siebert (2007) for a recent very useful discussion and an application. 

The issue of boundedness was discovered a relatively long time ago and received re- 
newed attention in the 1990s when the new growth theory was being developed. A more 
general treatment of this problem was undertaken by von Weizsäcker (1965) who compares 
utility levels in unbounded circumstances by using “overtaking criteria”. 

Expressions for explicit solutions for consumption have been known for a while. The 
case of a logarithmic utility function, i.e. where g = 1 and where the fraction in front of 
the curly brackets in (5.6.10) simplifies to p, was obtained by Blanchard (1985). 

Vissing-Jorgensen (2002) provides micro-evidence on the level of the intertemporal 
elasticity of substitution. 

Matching models of unemployment go back to Pissarides (1985). For a textbook 
treatment, see Pissarides (2000). 


5.8. Further reading and exercises 137 


Exercises chapter 5 
Applied Intertemporal Optimization 


Hamiltonians 


1. Optimal consumption over an infinite horizon 
Solve the maximization problem 


max f ek- lne(r)dr, 
CLT t 


subject to i 
p(t)c(t)h + A(t) =r(t)A(t)+w(t). 
by 


(a) using the present-value Hamiltonian. Compare the result to (5.6.5). 
(b) Use u(c(r)) instead of Inc(r). 


2. Adjustment costs 
Solve the adjustment cost example for 


®(1) =I. 


What do optimality conditions mean? What is the optimal end-point value for K 
and I? 


3. Consumption over the life cycle 
The utility of an individual, born at s and living for T periods is given at time t by 


s+T 
u(s, t) =] el" In (e(s,7)) dr. 
t 

The individual’s budget constraint is given by 

s+T 

f Dr(t) c(s,7) dt =h (s,t) +a (s, t) 

t 

where 
T s+T 
Dpr (T) = exp -f r (u) du , h(s,t)= I D, (7) w(s,7) dr. 
t t 


This deplorable individual would like to know how he can lead a happy life but, 
unfortunately, has not studied optimal control theory! 
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(a) What would you recommend him? Use a Hamiltonian approach and distinguish 
between changes of consumption and the initial level. Which information do 
you need to determine the initial consumption level? What information would 
you expect this individual to provide you with? In other words, which of the 
above maximization problems makes sense? Why not the other one? 


(b) Assume all prices are constant. Draw the path of consumption in a (t,c(t)) 
diagram. Draw the path of asset holdings a(t) in the same diagram, by guessing 
how you would expect it to look. (You could compute it if you want) 


4. Optimal consumption 


(a) Derive the optimal allocation of expenditure and consumption over time for 


c(r) 7 -1 
l-o 


u(e(7)) = 


, ¢ >0 


by employing the Hamiltonian. 

(b) Show that this function includes the logarithmic utility function for o = 1 
(apply L’Héspital’s rule). 

(c) Does the utility function u(c(t)) make sense for o > 1? Why (not)? 


(d) Compute the intertemporal elasticity of substitution for this utility function 
following the discussion after (5.6.4). What is the intertemporal elasticity of 
substitution for the logarithmic utility function u (c) = Inc? 


5. A central planner 
You are responsible for the future well-being of hundreds of million of Europeans 
and centrally plan the EU by assigning a consumption path to each inhabitant. 
Your problem consists in maximizing a social welfare function of the form 


U’ (t) = i: ere (œ= -1)(1- o) dr 


subject to the EU resource constraint 
K=BK-C (5.8.1) 


(a) What are optimality conditions? What is the consumption growth rate? 


(b) Under which conditions is the problem well defined (boundedness condition)? 
Insert the consumption growth rate and show under which conditions the utility 
function is bounded. Does this condition still allow for positive long-run growth 
rates? 


(c) What is the growth rate of the capital stock? Compute the initial consumption 
level, by using the no-Ponzi-game condition (5.4.4). 
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(d) Under which conditions could you resign from your job without making anyone 
less happy than before? 


6. Optimal consumption levels 


(a) Derive a rule for the optimal consumption level for a time-varying interest rate 
r (t). Show that (5.6.10) can be generalized to 


1 
c(t) = EA LY md Oe 
are: C=O as gy 


where W (t) is human wealth. 
(b) What does this imply for the wealth level a (t)? 


7. Investment and the interest rate 


(a) Use the result of 5 c) and check under which conditions investment is a de- 
creasing function of the interest rate. 


(b) Perform the same analysis for a budget constraint @ = ra + w — c instead of 
(5.8.1). 


8. The Ramsey growth model 
Consider a central planer with a standard CES utility function as in ex. 5. Let the 
resource constraint now be given by 


K=Y(K,L)-C-ôK 
where Y (.) is a neoclassical production function. 


(a) Draw a phase diagram. What is the long-run equilibrium? Perform a stability 
analysis graphically and analytically (locally). 


(b) Is the utility function bounded? 


9. An exam question 

Consider a decentralized economy in continuous time. Factors of production are 
capital and labour. The initial capital stock is Ko, labour endowment is L. Capital 
is the only asset, i.e. households can save only by buying capital. Capital can be 
accumulated also at the aggregate level, K (T) = I (r) — ôK (T). Households have a 
corresponding budget constraint and a standard intertemporal utility function with 
time preference rate p and infinite planning horizon. Firms produce under perfect 
competition. Describe such an economy in a formal way and derive its reduced form. 
Do this step by step: 


(a) Choose a typical production function. 
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(b) Derive factor demand functions by firms. 


(c) Let the budget constraints of households be given by 
a(t) =r(r)a(r) +w” (7) —e(r). 


Specify the maximization problem of a household and solve it. 


(d) Aggregate the optimal individual decision over all households and describe the 
evolution of aggregate consumption. 


(e) Formulate the goods market equilibrium. 


(f) Show that the budget constraint of the household is consistent with the aggre- 
gate goods market equilibrium. 


(g) Derive the reduced form 


Siz BGO) E 
K (r) =Y (7) -© (T) — 6K (7), So ao- p 


by going through these steps and explain the economics behind this reduced 
form. 


Chapter 6 


Infinite horizon again 


This chapter reanalyzes maximization problems in continuous time that are known from 
the chapter on Hamiltonians. It shows how to solve them with dynamic programming 
methods. The sole objective of this chapter is to present the dynamic programming 
method in a well-known deterministic setup such that its use in a stochastic world in 
subsequent chapters becomes more accessible. 


6.1 Intertemporal utility maximization 


We consider a maximization problem that is very similar to the introductory example for 
the Hamiltonian in section 5.1 or the infinite horizon case in section 5.3. Compared to 
5.1, the utility function here is more general and the planning horizon is infinity. None 
of this is important, however, for understanding the differences in the approach between 
the Hamiltonian and dynamic programming. 


6.1.1 The setup 
Utility of the individual is given by 


U (t)= [ ely, (e(r)) dr. (6.1.1) 


t 


Her budget constrained equates wealth accumulation with savings, 
å = ra + w — pe. (6.1.2) 


The individual can choose the path of consumption {c (T)} between now and infinity and 
takes prices and factor rewards as given. 


6.1.2 Solving by dynamic programming 


As in models of discrete time, the value V (a(t)) of the optimal program is defined by 
the maximum overall utility level that can be reached by choosing the consumption path 
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optimally given the constraint, V (a (t)) = maxy,;); U (t) subject to (6.1.2). When house- 
holds behave optimally between today and infinity by choosing the optimal consumption 
path {c(7)}, their overall utility U (t) is given by V (a(t)). 


e A prelude on the Bellman equation 


The derivation of the Bellman equation under continuous time is not as obvious as 
under discrete time. Even though we do have a today, t, we do not have a clear tomorrow 
(like a t+ 1 in discrete time). We therefore need to construct a tomorrow by adding a 
“small” time interval At to t. Tomorrow would then be t + At. Note that this derivation 
is heuristic and more rigorous approaches exist. See e.g. Sennewald (2007) for further 
references to the literature. 

Following Bellman’s idea, we rewrite the objective function as the sum of two subpe- 
riods, 


t+At 00 
U(t) = / elt, (c(7)) dr +f er-u (c(r)) dr, 


+At 
where At is a “small” time interval. As we did in discrete time, we exploit here the additive 
separability of the objective function. This is the first step of simplifying the maximization 
problem, as discussed for the discrete-time case after (3.3.4). When we approximate 
the first integral (think of the area below the function u(c(7)) plotted over time T) by 
u(c(t)) At and the discounting between t and t + At by TA and we assume that as of 
t + At we behave optimally, we can rewrite the value function V (a (t)) = maxy.,,); U (t) 
as 


c(t) 
The assumption of behaving optimally as of t + At can be seen formally in the fact 
that V (a(t + At)) now replaces U (t + At). This is the second step in the procedure 
to simplify the maximization problem. After these two steps, we are left with only one 
choice variable c (t) instead of the entire path {c (T)}.When we first multiply this expres- 


sion by 1 + pAt, then divide by At and finally move Vew) to the right hand side, we 


get pV (a(t)) = maxi) fu (c (t)) [1 + pAt] 4 eee . Taking the limit limayo 
gives the Bellman equation, 


V (a(t)) = max fu (c(t)) At + om (a(t + any} 


Wot) 


pV (a (t)) = max fu (c(t)) + Ti 


6.1.3 
u (6.1.3) 


This equation again shows Bellman’s trick: A maximization problem, consisting of the 
choice of a path of a choice variable, was broken down to a maximization problem where 
only the level of the choice variable in t has to be chosen. 

The structure of this equation can also be understood from a more intuitive perspec- 
tive: The term pV (a (t)) can best be understood when comparing it to rv, capital income 
at each instant of an individual who owns a capital stock of value v and the interest rate is 
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r. A household that behaves optimally “owns” the value V (a (t)) from optimal behaviour 
and receives a utility stream of pV (a (t)). This “utility income” at each instant is given by 
instantaneous utility from consumption plus the change in the value of optimal behaviour. 
Note that this structure is identical to the capital-market no-arbitrage condition (4.4.7), 
r (t)v (t) = a(t) + ù (t) - the capital income stream from holding wealth v (t) on a bank 
account is identical to dividend payments 7 (t) plus the change ù (t) in the market price 
when holding the same level of wealth in stocks. 

While the derivation just shown is the standard one, we will now present an alternative 
approach which illustrates the economic content of the Bellman equation and which is 
more straightforward. Given the objective function in (6.1.1), we can ask how overall 
utility U (t) changes over time. To this end, we compute the derivative dU (t) /dt and find 
(using the Leibniz rule 4.3.3 from ch. 4.3.1) 


Ù (t) = —e~ Pty (e (t)) +f L gruy (c(7)) dr = —u (c (t)) + pU (t). 
t 

Overall utility U (t) reduces as time goes by by the amount u(c(t)) at each instant 
(as the integral becomes “smaller” when current consumption in t is lost and we start 
an instant after t) and increases by pU (t) (as we gain because future utilities come 
closer to today when today moves into the future). Rearranging this equation gives 
pU (t) = u (e (t))+Ù (t). When overall utility is replaced by the value function, we obtain 
pV (a(t)) = u(c(t))+V (a(t)) which corresponds in its structure to the Bellman equation 
(6.1.3). 


e DP1: Bellman equation and first-order conditions 


We will now follow the three-step procedure to maximization when using the dynamic 
programming approach as we got to know it in discrete time setups is section 3.3. When 
we compute the Bellman equation for our case, we obtain for the derivative in (6.1.3) 
dV (a(t)) /dt = V’ (a(t)) @ which gives with the budget constraint (6.1.2) 


pV (a(t)) = m {u (ce (t)) + V’ (a (t)) [ra + w — pe]}. (6.1.4) 


The first-order condition reads 


u' (c(t)) = pV’ (a (t)) (6.1.5) 


and makes consumption a function of the state variable, c(t) = c(a(t)). In contrast to 
discrete time models, there is no tomorrow and the interest rate and the time preference 
rate, present for example in (3.4.5), are absent here. This first-order condition also cap- 
tures “pros and cons” of more consumption today. The advantage is higher instantaneous 
utility, the disadvantage is the reduction in wealth. The disadvantage is captured by the 
change in overall utility due to changes in a(t), i.e. the shadow price V” (a (t)) , times the 
price of one unit of consumption in units of the capital good. Loosely speaking, when con- 
sumption goes up today by one unit, wealth goes down by p units. Higher consumption 
increases utility by wu’ (c(t)), p units less wealth reduces overall utility by pV’ (a (t)). 
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e DP2: Evolution of the costate variable 


In continuous time, the second step of the dynamic programming approach to maxi- 
mization can be subdivided into two substeps. (i) In the first, we look at the maximized 
Bellman equation, 


pV (a) = u (c (a)) + V’ (a) [ra + w — pe (a)] . 


The first-order condition (6.1.5) together with the maximized Bellman equation deter- 
mines the evolution of the control variable c(t) and V(a(t)). This system can be used 
as a basis for numerical solution. Again, however, the maximized Bellman equation does 
not provide very much insight from an analytical perspective. Computing the derivative 
with respect to a(t) however (and using the envelope theorem) gives an expression for 
the shadow price of wealth that will be more useful, 


pV' (a) = V" (a) [ra + w — pc] + V’ (a)r es (6.1.6) 
(p-r) V' (a) = V” (a) [ra + w — pe]. 


(ii) In the second step, we compute the derivative of the costate variable V’ (a) with 
respect to time, giving 


where the last equality used (6.1.6). Dividing by V’ (a) and using the usual notation 
V’ (a) = dV’ (a) /dt, this can be written as 
al 
V’ (a) er 
V’ (a) 


(6.1.7) 


This equation describes the evolution of the costate variable V’ (a), the shadow price of 
wealth. 


e DP3: Inserting first-order conditions 


The derivative of the first-order condition with respect to time is given by (apply first 
logs) 
n : al 
UONE IRMO 
u 


AD. Via) 


Inserting (6.1.7) gives 


A A H l 
2 C ogy diy ot Chae eee 
u (c) p u 


This is the well-known Keynes Ramsey rule. 
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6.2 Comparing dynamic programming to Hamiltoni- 
ans 
When we compare optimality conditions under dynamic programming with those obtained 


when employing the Hamiltonian, we find that they are identical. Observing that our 
costate evolves when using dynamic programming according to 


V’ (a) 
V” (a) 


=p—?, 


as just shown in (6.1.7), we obtain the same equation as we had in the Hamiltonian 
approach for the evolution of the costate variable, see e.g. (5.1.5) or (5.6.13). Comparing 
first-order conditions (5.1.4) or (5.2.9) with (6.1.5), we see that they would be identical if 
we had chosen exactly the same maximization problem. This is not surprising given our 
applied view of optimization: If there is one optimal path that maximizes some objective 
function, this one path should always be optimal, independently of which maximization 
procedure is chosen. 

A comparison of optimality conditions is also useful for an alternative purpose, how- 
ever. As (6.1.7) and e.g. (5.1.5) or (5.6.13) are identical, we can conclude that the deriv- 
ative V’ (a(t)) of the value function with respect to the state variable, in our case a, is 
identical to the costate variable A in the current-value Hamiltonian approach, V’ (a) = A. 
This is where the interpretation for the costate variable in the Hamiltonian approach in 
ch. 5.2.3 came from. There, we said that the costate variable A stands for the increase in 
the value of the optimal program when an additional unit of the state variable becomes 
available; this is exactly what V” (a) stands for. Hence, the interpretation of a costate 
variable A is similar to the interpretation of the Lagrange multiplier in static maximization 
problems. 


6.3 Dynamic programming with two state variables 


As a final example for maximization problems in continuous time that can be solved 
with dynamic programming, we look at a maximization problem with two state variables. 
Think e.g. of an agent who can save by putting savings on a bank account or by ac- 
cumulating human capital. Or think of a central planner who can increase total factor 
productivity or the capital stock. We look here at the first case. 

Our agent has a standard objective function, 


U (t) = [ elu (e(T)) dr. 


It is maximized subject to two constraints. They describe the evolution of the state 
variables wealth a and human capital h, 


å = f (a,h,c), h = g(a,h,c). (6.3.1) 


146 Chapter 6. Infinite horizon again 


We do not give explicit expressions for the functions f (.) or g(.) but one can think 
of a standard resource constraint for f (.) as in (6.1.2) and a functional form for g (.) 
that captures a trade-off between consumption and human capital accumulation: Human 
capital accumulation is faster when a and h are large but decreases in c. To be precise, 
we assume that both f (.) and g (.) increase in a and h but decrease in c. 


e DP1: Bellman equation and first-order conditions 


In this case, the Bellman equation reads 


pV (a,h) = max fu (c) + Tar) = max {u (c) + Vaf (.) + Vag ()}- 
There are simply two partial derivatives of the value function after the u (c) term times 
the da and dh term, respectively, instead of one as in (6.1.4), where there is only one state 
variable. 

Given that there is still only one control variable, consumption, there is only one 
first-order condition. This is clearly specific to this example. One could think of a time 
constraint for human capital accumulation (a trade-off between leisure and learning - 
think of the Lucas (1988) model) where agents choose the share of their time used for 
accumulating human capital. In this case, there would be two first-order conditions. Here, 
however, we have just the one for consumption, given by 


u' (c) 4 a | 29 = 0 (6.3.2) 


When we compare this condition with the one-state-variable case in (6.1.5), we see 
that the first two terms wu’ (c) + V, 280 correspond exactly to (6.1.5): If we had specified 
f (.) as in the budget constraint (6.1.2), the first two terms would be identical to (6.1.5). 
The third term Vp 22 is new and stems from the second state variable: Consumption now 
not only affects the accumulation of wealth but also the accumulation of human capital. 
More consumption gives higher instantaneous utility but, at the same time, decreases 
future wealth and - now new - the future human capital stock as well. 


e DP2: Evolution of the costate variables 


As always, we need to understand the evolution of the costate variable(s). In a setup 
with two state variables, there are two costate variables, or, economically speaking, a 
shadow price of wealth a and a shadow price of human capital h. This is obtained by 
partially differentiating the maximized Bellman equation, first with respect to a, then 
with respect to h. Doing this, we get (employing the envelope theorem right away) 


Va = Vaal () + VoL + Vrag () +WEU, Pom 
PVn = Vanf (.) + Va oe Ked ONETU a ea 
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As in ch. 6.1, the second step of DP2 consists of computing the time derivatives of the 
costate variables and in reinserting (6.3.3) and (6.3.4). We first compute time derivatives, 
inserting (6.3.1) into the last step, 


dV, sib A } 
Aini = Vaad + Vanh = Vaa f (.) + Vang (.) : 
dV; „h A } 
Mt = Vha + Vinh = Vha f (-) + Vang (-). 


Then inserting (6.3.3) and (6.3.4), we find 


dV, (a,h) _ att.) Il) 
dt = PMaim Va Oa Vn Oa ’ 
dV, (ah)_ ,, 1, OF), O90) 
e ad a er 


The nice feature of this last step is the fact that the cross derivatives Van and Vha “disap- 
pear”, i.e. can be substituted out again, using the fact that f,, (v,y) = fya (£, y) for any 
twice differentiable function f (x,y). Writing these equations as 


Vv. Of (.) _ Vn Og.) 
Va Oa V, ôa ’ 
Vi VaOf(.) O 


yo ðh ah” 


allows us to give an interpretation that links them to the standard one-state case in (6.1.7). 
The costate variable V, evolves as above, only that instead of the interest rate, we find 
here oft) + Va 96), The first derivative Of (.) /Oa captures the effect a change in a has 
on the first constraint; in fact, if f (.) represented a budget constraint as in (6.1.2), this 
would be identical to the interest rate. The second term 0g (.) /Oa captures the effect 
of a change in a on the second constraint; by how much would h increase if there was 
more a? This effect is multiplied by V;,/V., the relative shadow price of h. An analogous 
interpretation is possible for Va IVh. 


e DP3: Inserting first-order conditions 


The final step consists of computing the derivative of the first-order condition (6.3.2) 
with respect to time and replacing the time derivatives V, and V, by the expressions 
from the previous step DP2. The principle of how to obtain a solution therefore remains 
unchanged when having two state variables instead of one. Unfortunately, however, it is 
generally not possible to eliminate the shadow prices from the resulting equation which 
describes the evolution of the control variable. Some economically suitable assumptions 
concerning f (.) or g(.) could, however, help. 
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6.4 Nominal and real interest rates and inflation 


We present here a simple model which allows us to understand the difference between the 
nominal and real interest rate and the determinants of inflation. This chapter is intended 
(i) to provide another example where dynamic programming can be used and (ii) to show 
again how to go from a general decentralized description of an economy to its reduced 
form and thereby obtain economic insights. 


6.4.1 Firms, the central bank and the government 


e Firms 


Firms use a standard neoclassical technology Y = Y (K, L). Producing under perfect 
competition implies factor demand function of 


OY OY 
K _ — 
~ POR’ ~Ha Oe) 


e The central bank and the government 


Studying the behaviour of central banks will fill a lot of books. We present the be- 
haviour of the central bank in a very simple way - so simple that what the central bank 
does here (buy bonds directly from the government) is actually illegal in most OECD 
countries. Despite this simple presentation, the general result we obtain later would hold 
in more realistic setups as well. 

The central bank issues money M (t) in exchange for government bonds B (t). It 
receives interest payments iB from the government on the bonds. The balance of the 
central bank is therefore iB + M = B. This equation says that bond holdings by the 

central bank increase by B, either when the central bank issues money M or receives 
interest payments 7B on hands it holds. 

The government’s budget constraint reads G + iB = T + B. General government 
expenditure G plus interest payments 7B on government debt B is financed by tax income 
T and deficit B. We assume that only the central bank holds government bonds and not 
private households. Combining the government with the central bank budget therefore 
yields 


M=G-T. 
This equation says that an increase in monetary supply is either used to finance govern- 


ment expenditure minus tax income or, if G = 0 for simplicity, any monetary increase is 
given to households in the form of negative taxes, T = —M. 
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6.4.2 Households 


Household preferences are described by a “money-in-utility” function, 


f * ezel [in c(t) +yIn a 1 dr. 


In addition to utility from consumption c(t), utility is derived from holding a certain 
stock m (t) of money, given a price level p(t). Given that wealth of households consists of 
capital goods plus money, a (t) = k (t) +m (t), and that holding money pays no interest, 
the household’s budget constraint can be shown to read (see exercises) 


å =iļa — m] +w- T/L- pe. 


Tax payments T/L per representative household are lump-sum. If taxes are negative, 
T/L represents transfers from the government to households. The interest rate 7 is defined 
according to 


When households choose consumption and the amount of money held optimally, con- 
sumption growth follows (see exercises) 


Money demand is given by (see exercises) 


pe 


6.4.3 Equilibrium 


e The reduced form 


Equilibrium requires equality of supply and demand on the goods market. This is 
obtained if total supply Y equals demand ČC + J. Letting capital accumulation follow 
K =I — ôK, we get 

K =Y (K,L)-C - ôK. (6.4.2) 


This equation determines K. As capital and consumption goods are traded on the same 
market, this equation implies v = p and the nominal interest rate becomes with (6.4.1) 


wE p oY 


p 
p p OK p 


The nominal interest rate is given by marginal productivity of capital w*/p = 0Y /ƏK 
(the “real interest rate”) plus inflation p/p. 
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Aggregating over households yields aggregate consumption growth of C /C =i-p/p- 
p. Inserting (6.4.3) yields 


C 0r 
E .4.4 
CIK’ (6.4.4) 
Aggregate money demand is given by 
ma (6.4.5) 
i 


Given an exogenous money supply rule and appropriate boundary conditions, these four 
equations determine the paths of K, C, i and the price level p. 

One standard property of models with flexible prices is the dichotomy between real 
variables and nominal variables. The evolution of consumption and capital - the real 
side of the economy - is completely independent of monetary influences: Equation (6.4.2) 
and (6.4.4) determine the paths of K and C just as in the standard optimal growth 
model without money - see (5.6.12) and (5.6.14) in ch. 5.6.3. Hence, when thinking 
about equilibrium in this economy, we can think about the real side on the one hand 
- independently of monetary issues - and about the nominal side on the other hand. 
Monetary variables have no real effect but real variables have an effect on monetary 
variables like e.g. inflation. 

Needless to say that the real world does not have perfectly flexible prices such that 
one should expect monetary variables to have an impact on the real economy. This model 
is therefore a starting point to well understanding structures and not a fully developed 
model for analysing monetary questions in a very realistic way. Price rigidity would have 
to be included before doing this. 


e A steady state 


Assume the technology Y (K, L) is such that in the long-run K is constant. As a 
consequence, aggregate consumption C is constant as well. Hence, with respect to real 
variables (including, in addition to K and C, the real interest rate and output), we are in 
a steady state as in ch. 5.6.3. 

Depending on exogenous money supply, equations (6.4.3) and (6.4.5) determine the 
price level and the nominal interest rate. Substituting the nominal interest rate out, we 


obtain 
p pe z OY 


p 'M OK’ 


This is a differential equation which is plotted in the next figure. As this figure shows, 
provided that M is constant, there is a price level p* which implies that there is zero 
inflation. 
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Figure 6.4.1 The price level p* in a monetary economy 


Now assume there is money growth, M > 0. The figure then shows that the price level 
p* increases as the slope of the line becomes flatter. Money growth in an economy with 
constant GDP implies inflation. By looking at (6.4.3) and (6.4.5) again and by focusing 
on equilibria with constant inflation rates, we know from (6.4.3) that a constant inflation 
rate implies a constant nominal interest rate. Hence, by differentiating the equilibrium 
(6.4.5) on the money market, we get 


Be (6.4.6) 


In an economy with constant GDP and increasing money supply, the inflation rate is 
identical to the growth rate of money supply. 


e A growth equilibrium 


Now assume there is (exogenous) technological progress at a rate g such that in the 
long run Y/Y = C/C = K/K = g. Then by again assuming a constant inflation rate 
(implying a constant nominal interest rate) and going through the same steps that led to 
(6.4.6), we find by differentiating (6.4.5) 


Inflation is given by the difference between the growth rate of money supply and con- 
sumption growth. 
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e Exogenous nominal interest rates 


The current thinking about central bank behaviour differs from the view that the 
central bank chooses money supply M as assumed so far. The central bank rather sets 
nominal interest rates and money supply adjusts (where one should keep in mind that 
money supply is more than just cash used for exchange as modelled here). Can nominal 
interest rate setting be analyzed in this setup? 

Equilibrium is described by equations (6.4.2) to (6.4.5). They were understood to 
determine the paths of K, C, i and the price level p, given a money supply choice M by 
the central bank. If one believes that nominal interest rate setting is more realistic, these 
four equations would simply determine the paths of K, C, M and the price level p, given a 
nominal interest rate choice i by the central bank. Hence, simply by making an exogenous 
variable, MW, endogenous and making a previously endogenous variable, i, exogenous, the 
same model can be used to understand the effects of higher and lower nominal interest 
rates on the economy. 

Due to perfect price flexibility, real quantities remain unaffected by the nominal in- 
terest rate. Consumption, investment, GDP, the real interest rate, real wages are all 
determined, as before, by (6.4.2) and (6.4.4) - the dichotomy between real and nominal 
quantities continues given price flexibility. In (6.4.3), a change in the nominal interest rate 
affects inflation: high (nominal) interest rates imply high inflation, low nominal interest 
rates imply low inflation. From the money market equilibrium in (6.4.5) one can then 
conclude what this implies for money supply, again both for a growing or a stationary 
economy. Much more needs to be said about these issues before policy implications can 
be discussed. Any analysis in a general equilibrium framework would however partly be 
driven by the relationships presented here. 


6.5 Further reading and exercises 


An alternative way, which is not based on dynamic programming, to reach the same 
conclusion about the interpretation of the costate variable for Hamiltonian maximization 
as here in ch. 6.2 is provided by Intriligator (1971, p. 352). An excellent overview and 
introduction to Monetary economics is provided by Walsh (2003). 
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Exercises chapter 6 
Applied Intertemporal Optimization 


Dynamic Programming in Continuous Time 


1. The envelope theorem once again 
Compute the derivative (6.1.6) of the maximized Bellman equation without using 
the envelope theorem. 


2. A firm with adjustment costs 
Consider again, as in ch. 5.5.1, a firm with adjustment cost. The firm’s objective is 


max | eq (7) dr. 


HELO Jt 


In contrast to ch. 5.5.1, the firm now has an infinite planning horizon and employs 
two factors of production, capital and labour. Instantaneous profits are 


r =pF(K,L)—-wL-I-al’, 


where investment J also comprises adjustment costs for a > 0. Capital, owned by 
the firm, accumulates according to K = [—dK. All parameters ô, a, 8 are constant. 


(a) Solve this maximization problem by using the dynamic programming approach. You 


may choose appropriate (numerical or other) values for parameters where this 
simplifies the solution (and does not destroy the spirit of this exercise). 


(b) Show that in the long-run with adjustment costs and at each point in time un- 
der the absence of adjustment costs, capital is paid its value marginal product. 
Why is labour being paid its value marginal product at each point in time? 


3. Money in the utility function 
Consider an individual with the following utility function 


m(r) 


p(T) 


As always, p is the time preference rate and c(r) is consumption. This utility 
function also captures demand for money by including a real monetary stock of 


dr. 


U (t) = l e Plt] in c(7)+yln 
t 
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m (T) /p(T) in the utility function where m (T) is the amount of cash and p (T) is 
the price level of the economy. Let the budget constraint of the individual be 


4=ila—m|+w-—T/L — pe. 


where a is the total wealth consisting of shares in firms plus money, a = k + m and 
i is the nominal interest rate. 


(a) Derive the budget constraint by assuming interest payments of i on shares in 
firms and zero interest rates on money. 


(b) Derive the optimal money demand. 


4. Nominal and real interest rates in general equilibrium 
Put households from exercise 3 in general equilibrium with capital accumulation and 
a central bank which chooses money supply M. Compute the real and the nominal 
interest rate in a long-run equilibrium. 
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6.6 Looking back 


This is the end of part I and II. This is often also the end of a course. This is a good 
moment to look back at what has been accomplished. After 14 or 15 lectures and the 
same number of exercise classes, the amount of material covered is fairly impressive. 

In terms of maximization tools, this first part has covered 


e Solving by substitution 
e Lagrange methods in discrete and continuous time 
e Dynamic programming in discrete time and continuous time 


e Hamiltonian 
With respect to model building components, we have learnt 


e how to build budget constraints 
e how to structure the presentation of a model 


e how to derive reduced forms 
From an economic perspective, the first part presented 


e the two-period OLG model 

e the optimal saving central planner model in discrete and continuous time 
e the matching approach to unemployment 

e the decentralized optimal growth model and 


e an optimal growth model with money 


Most importantly, however, the tools presented here allow students to “become in- 
dependent”. A very large part of the Economics literature (acknowledging that game 
theoretic approaches have not been covered here at all) is now open and accessible and 
the basis for understanding a paper in detail (and not just the overall argument) and for 
presenting their own arguments in a scientific language are laid out. 

Clearly, models with uncertainty present additional challenges. They will be presented 
and overcome in part III and part IV. 
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Part III 


Stochastic models in discrete time 
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In part III, the world becomes stochastic. Parts I and II provided many optimization 
methods for deterministic setups, both in discrete and continuous time. All economic 
questions that were analyzed were viewed as “sufficiently deterministic”. If there was 
any uncertainty in the setup of the problem, we simply ignored it or argued that it is of 
no importance for understanding the basic properties and relationships of the economic 
question. This is a good approach to many economic questions. 

Generally speaking, however, real life has few certain components. Death is certain, 
but when? Taxes are certain, but how high are they? We know that we all exist - but 
don’t ask philosophers. Part III (and part IV later) will take uncertainty in life seriously 
and incorporate it explicitly in the analysis of economic problems. We follow the same 
distinction as in part I and II - we first analyse the effects of uncertainty on economic 
behaviour in discrete time setups in part III and then move to continuous time setups in 
part IV. 

Chapter 7 and 8 are an extended version of chapter 2. As we are in a stochastic world, 
however, chapter 7 will first spend some time reviewing some basics of random variables, 
their moments and distributions. Chapter 7 also looks at difference equations. As they 
are now stochastic, they allow us to understand how distributions change over time and 
how a distribution converges - in the example we look at - to a limiting distribution. 
The limiting distribution is the stochastic equivalent to a fix point or steady state in 
deterministic setups. 

Chapter 8 looks at maximization problems in this stochastic framework and focuses on 
the simplest case of two-period models. A general equilibrium analysis with an overlapping 
generations setup will allow us to look at the new aspects introduced by uncertainty 
for an intertemporal consumption and saving problem. We will also see how one can 
easily understand dynamic behaviour of various variables and derive properties of long- 
run distributions in general equilibrium by graphical analysis. One can for example easily 
obtain the range of the long-run distribution for capital, output and consumption. This 
increases intuitive understanding of the processes at hand tremendously and helps a lot as 
a guide to numerical analysis. Further examples include borrowing and lending between 
risk-averse and risk-neutral households, the pricing of assets in a stochastic world and a 
first look at ‘natural volatility’, a view of business cycles which stresses the link between 
jointly endogenously determined short-run fluctuations and long-run growth. 

Chapter 9 is then similar to chapter 3 and looks at multi-period, i.e. infinite horizon, 
problems. As in each chapter, we start with the classic intertemporal utility maximization 
problem. We then move on to various important applications. The first is a central planner 
stochastic growth model, the second is capital asset pricing in general equilibrium and how 
it relates to utility maximization. We continue with endogenous labour supply and the 
matching model of unemployment. The next section then covers how many maximization 
problems can be solved without using dynamic programming or the Lagrangian. In fact, 
many problems can be solved simply by inserting, despite uncertainty. This will be 
illustrated with many further applications. A final section on finite horizons concludes. 


160 


Chapter 7 


Stochastic difference equations and 
moments 


Before we look at difference equations in section 7.4, we will first spend a few sections 
reviewing basic concepts related to uncertain environments. These concepts will be useful 
at later stages. 


7.1 Basics on random variables 


Let us first have a look at some basics of random variables. This follows Evans, Hastings 
and Peacock (2000). 


7.1.1 Some concepts 


A probabilistic experiment is an occurrence where a complex natural background leads to 
a chance outcome. The set of possible outcomes of a probabilistic experiment is called the 
possibility space. A random variable (RV) X is a function which maps from the possibility 
space into a set of numbers. The set of numbers this RV can take is called the range of 
this variable X. 

The distribution function F associated with the RV X is a function which maps from 
the range into the probability domain [0,1], 


F (x) = Prob (X < z). 


The probability that X has a realization of x or smaller is given by F (x). 

We now need to make a distinction between discrete and continuous RVs. When the 
RV X has a discrete range then f (x) gives finite probabilities and is called the probability 
function or probability mass function. The probability that X has the realization of x is 


given by f (x). 
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When the RV X is continuous, the first derivative of the distribution function F 


ta) = #2 


is called the probability density function f. The probability that the realization of X lies 
between, say, a and b > a is given by F (b) — F (a) = T f (a) dx. Hence the probability 
that X equals a is zero. 


7.1.2 An illustration 


e Discrete random variable 


Consider the probabilistic experiment ’tossing a coin twice’. The possibility space is 
given by {HH, HT, TH, TT}. Define the RV "Number of heads’. The range of this 
variable is given by {0,1,2}. Assuming that the coin falls on either side with the same 
probability, the probability function of this RV is given by 


20 0 
P(e) ak 5 fr eae 1 
.25 2 


e Continuous random variable 


Think of next weekend. You might consider going to a pub to meet friends. Before 
you go there, you do not know how much time you will spend there. If you meet a lot of 
friends, you will stay longer; if you drink just one beer, you will leave soon. Hence, going 
to a pub on a weekend is a probabilistic experiment with a chance outcome. 

The set of possible outcomes with respect to the amount of time spent in a pub is the 
possibility space. Our random variable T maps from this possibility space into a set of 
numbers with a range from 0 to, let’s say, 4 hours (as the pub closes at 1 am and you never 
go there before 9 p.m.). As time is continuous, T € [0,4] is a continuous random variable. 
The distribution function F (t) gives you the probability that you spend a period of length 
t or shorter in the pub. The probability that you spend between 1.5 and two hours in the 
pub is given by T f (t) dt, where f (t) is the density function f (t) = dF (t) /dt. 


7.2 Examples for random variables 


We now look at some examples of RVs that are useful for later applications. As an RV 
is completely characterized by its range and its probability or density function, we will 
describe RVs by providing this information. Many more random variables exist than those 
presented here and the interested reader is referred to the “further reading” section at the 
end. 
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7.2.1 Discrete random variables 


e Discrete uniform distribution 


range x E€ {a,a+l,...,b—1,5} 
probability function | f (x) =1/(b— a+ 1) 


An example for this RV is the die. Its range is 1 to 6, the probability for any number 
(at least for fair dice) is 1/6. 


e The Poisson distribution 


range x € {0, 1, 2,...} 


probability function | f (x) = mae 


x! 


Here is some positive parameter. When we talk about stochastic processes in part 
IV, this will be called the arrival rate. An example for this RV is e.g. the number of falling 
stars visible on a warm summer night at a nice beach. 


7.2.2 Continuous random variables 


e Normal distribution 


range x € ]—00, +00[ 


Bae 
density function | f (x) = age a 


The mean and the standard deviation of X are given by p and ø. 
e Standard normal distribution 


This is the normal distribution with mean and standard deviation given by u = 0 and 
o=1. 


e Exponential distribution 


range x € [0, cof 


density function | f (x) = \e~** 


Again, is some positive parameter. One standard example for x is the duration of 
unemployment for an individual who just lost his or her job. 
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7.2.3  Higher-dimensional random variables 


So far we have studied one-dimensional RVs. In what follows, we will occasionally work 
with multi-dimensional RVs as well. For our illustration purposes here it will suffice to 
focus on a two-dimensional normal distribution. Consider two random variables X and 
Xə. They are (jointly) normally distributed if the density function of X, and Xə is given 
by 


1 1 (27 2př12 z2) 
f (£1, £2) = e PN : 7.2.1 
(71,2) 2701029\/1 — p? l ) 
where č; = ee Pr: p= Zie (7.2.2) 
Oi 0102 


The mean and standard deviation of the RVs are denoted by u; and o;. The parameter 
p is called the correlation coefficient between X; and X> and is defined as the covariance 
O12 divided by the standard deviations (see ch. 7.3.1). 

The nice aspects about this two-dimensional normally distributed RV (the same holds 
for n-dimensional RVs) is that the density function of each individual RV X;, i.e. the mar- 
ginal density, is given by the standard expression which is independent of the correlation 


coefficient, 
1 


y af; 

f (xi) Jonah” (7.2.3) 
This implies a very convenient way to go from independent to correlated RVs in a multi- 
dimensional setting: When we want to assume independent normally distributed RVs, we 
assume that (7.2.3) holds for each random variable X; and set the correlation coefficient 
to zero. When we want to work with dependent RVs that are individually normally 
distributed, (7.2.3) holds for each RV individually as well but, in addition, we fix a non- 
zero coefficient of correlation p. 


7.3 Expected values, variances, covariances and all 
that 


Here we provide various definitions, some properties and results on transformations of 
RVs. Only in some selected cases do we provide proofs. A more in depth introduction 
can be found in many textbooks on statistics. 


7.3.1 Definitions 


For definitions, we shall focus on continuous random variables. For discrete random 
variables, the integral is replaced by a sum - in very loose notation, i g (x) dx is replaced 
by =2_,9 (ai), where a and b are constants which can be minus or plus infinity and g (x) 


is some function. In the following definitions, f .dz means the integral over the relevant 
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range (i.e. from minus to plus infinity or from the lower to upper bound of the range of 
the RV under consideration). 


Mean EX = [af (x) dx 
z =g (z, y) > EZ = J fg (x, y) f (x, y) dx dy 
Variance varX = f (x — EX)? f (x) dz 


kth uncentered moment EX* = fx"f (x) dr 
kth centered moment E(X SEX) =f (= fF (ede 


covariance cov(X,Y) = f f (x — EX) (y — EY) f (x, y) dx dy 
correlation coefficient Pxy = cov (X,Y) /vvarX varY 
independence p(XEA,YEB)=P(XEAP(YEB) 


Table 7.3.1 Some basic definitions 


7.3.2 Some properties of random variables 


e Basic 


Here are some useful properties of random variables. They are listed here for later 
reference. More background can be found in many statistics textbooks. 


E |a +bX]| = a +bEX 
E [bX + cY] = bEX + cEY 
E (XY) = EXEY +cov(X, Y) 


Table 7.3.2 Some properties of expectations 


varX = E |(X — EX)’] = E [X? — 2XEX + (EX)’| 
= EX? — 2 (EX? + (EX? = EX? — (EX? (7.3.1) 


var (a+ bX) = bvarX 
var (X +Y) = varX + varY + 2cov (X, Y) (7.3.2) 


Table 7.3.3 Some properties of variances 


cov (X, X) = varX 
cov (X,Y) = E (XY) — EXEY (7.3.3) 
cov (a +bX,c + dY) = bd cov (X, Y) 
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Table 7.3.4 Some properties of covariances 
e Advanced 


Here we present a theorem which is very intuitive and highly useful for analytically 
studying the pro- and countercyclical behaviour of endogenous variables in models of 
business cycles. The theorem says that if two variables depend “in the same sense” on 
some RV (i.e. they both increase or decrease in the RV), then these two variables have a 
positive covariance. If e.g. both GDP and R&D expenditure increase in TFP and TFP is 
random, then GDP and R&D expenditure are procyclical. 


Theorem 7.3.1 Let X be a random variable and f (X) and g(X) two functions such 
that F'(X) g' (X) = OVaeX. Then 


coul f (X),g(X)) 0. 


Proof. We only prove the “> 0” part. We know from (7.3.3) that cov(Y,Z) = 
E(YZ)-— EY EZ. With Y = f (X) and Z = g (X), we have 


cov (f (X), g(X)) = E(f (X)g (X)) — Ef (X) Eg (X) 
= ff (x) g(x) p(x) dx — ff (x) p(x) dxf g(x) p(x) dz, 


where p(x) is the density of X. Hence 


cov (f (X),g(X)) > 0 Jf (x) g(x) p(X) da > ff (x) p(x) dxf g (a) p (a) de. 


The last inequality holds for f'(X)g' (X) > 0 as shown by CebySev and presented in 
Mitrinovié (1970, Theorem 10, sect. 2.5, p. 40). m 


7.3.3 Functions on random variables 


We will occasionally encounter the situation where we need to compute density functions 
of functions of RVs. Here are some examples. 


e Linearly transforming a normally distributed RV 


Consider a normally distributed RV X ~ N (,07). What is the distribution of the 
Y =a+bX? We know from ch. 7.3.2 that for any RV, F(a +bX) = a+bEX and 
Var (a+ bX) = b?Var (X). As it can be shown that a linear transformation of a normally 
distributed RV gives a normally distributed RV again, Y is also normally distributed with 
Y ~ N (a+ bp, b?07). 
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e An exponential transformation 


Consider the RV Y = e* where X is again normally distributed. The RV Y is then 
lognormally distributed. A variable is lognormally distributed if its logarithm is normally 
distributed, In Y ~ N (1,07). The mean and variance of this distribution are given by 


i= etta? o = erto? (e” — 1) . (7.3.4) 
Clearly, Y can only have non-negative realizations. 
e Transformations of lognormal distributions 


Let there be two lognormally distributed variables Y and Z. Any transformation of 
the type Y°, where a is a constant, or products like Y Z are also lognormally distributed. 

To show this, remember that we can express Y and Z as Y = e*! and Z = e%?, 
with the X; being (jointly) normally distributed. Hence, for the first example, we can 
write Y° = e°%1, As aX; is normally distributed, Y® is lognormally distributed. For the 
second, we write YZ = e*1te*2 = eř1+%2, As the X; are (jointly) normally distributed, 
their sum is as well and Y Z is lognormally distributed. 

Total factor productivity is sometimes argued to be lognormally distributed. Its log- 
arithm is then normally distributed. See e.g. ch. 8.1.6. 


e The general case 


Consider now a general transformation of the type y = y (x) where the RV X has a 
density f (x). What is the density of Y? The answer comes from the following 


Theorem 7.3.2 Let X be a random variable with density f (x) and range [a,b] which 


can be | — œ, +00]. Let Y be defined by the monotonically increasing function y = y (a). 


Then the density g (y) is given by g (y) = f (x (y)) i on the range [y(a), y(b)]. 


a T b X 
Figure 7.3.1 Transforming a random variable 
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This theorem can be easily proven as follows. The proof is illustrated in fig. 7.3.1. 
The figure plots the RV X on the horizontal and the RV Y on the vertical axis. A 
monotonically increasing function y(x) represents the transformation of realizations x 
into y. 

Proof. The transformation of the range is immediately clear from the figure: When X 
is bounded between a and b, Y must be bounded between y (a) and y (b) . The proof for the 
density of Y requires a few more steps: The probability that y is smaller than some y (Z) is 
identical to the probability that X is smaller than this x. This follows from the monotonic- 
ity of the function y (x). As a consequence, the distribution function (cumulative density 
function) of Y is given by G (y) = F (x) where y = y (x) or, equivalently, x = x (y). The 


derivative then gives the density function, g (y) = Ge (y) = ae (x (y)) = f (x (y)) a E 


7.4 Examples of stochastic difference equations 


We now return to the first main objective of this chapter, the description of stochastic 
processes through stochastic difference equations. 


7.4.1 A first example 
e The difference equation 
Possibly the simplest stochastic difference equation is the following 
Li = ar-14+ Et, (7.4.1) 


where a is a positive constant and the stochastic component €+ is distributed according 
to some distribution function over a range which implies a mean u and a variance o°, 
c ~ (1,07). We do not make any specific assumption about £; at this point. Note 
that the stochastic components ¢; are i.i.d. (identically and independently distributed) 
which implies that the covariance between any two distinct €; is zero, cov(<;,¢,) = 0 
Vt Æ s. An alternative representation of x, with identical distributional properties would 
be x; = ar1 + u + v with v ~ (0,07). 


e Solving by substitution 


In complete analogy to deterministic difference equations in ch. 2.5.3, equation (7.4.1) 
can be solved for x; as a function of time and past realizations of £+, provided we have a 
boundary condition zo for t = 0. By repeated reinserting, we obtain 


2 
U1 = ax%+ €1, Lo = a [azo + €1] + E2 = a To + ae + €2, 


%3=4 [ax +a£ı + E2] + E3 = azo H Gey + GE2 + E3 
and eventually 


Ti = at £o + atte + at? Eg Sen HEt 
= azo + Ni_a" “es. (7.4.2) 
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e The mean 


A stochastic difference equation does not predict how the stochastic variable x; for 
t > 0 actually evolves, it only predicts the distribution of x, for all future points in time. 
The realization of x; is random. Hence, we can only hope to understand something about 
the distribution of x,;. To do so, we start by analyzing the mean of x, for future points in 
time. 

Denote the conditional expected value of x, by 7+, 


on = Egz, (7.4.3) 


i.e. čis the value expected when we are in t = 0 for x+ int > 0. The expectations operator 
Ego is conditional, i.e. it uses our “knowledge” in 0 when we compute the expectation for 
xı. The “0” says that we have all information for t = 0 and know therefore x9 and €£oọ, 
but we do not know €1, €2, etc. Put differently, we look at the conditional distribution of 
xı: What is the mean of x; conditional on zo? 

By applying the expectations operator Eo to (7.4.1), we obtain 


Tı —O0 ia + h- 
This is a deterministic difference equation which describes the evolution of the expected 
value of x; over time. There is again a standard solution to this equation which reads 
1 — af 
l-a’ 


S 


£, = azo + pdi_,a®* = a to + u (7.4.4) 


where we used Xt jats = a? +a! +... +a! = Etla’ and, from ch. 2.5.1, X? a* = 


(1 —a”tt)/(1— a). This equation shows that the expected value of x; changes over time 
as t changes. Note that <, might increase or decrease (see the exercises). 


e The variance 


Let us now look at the variance of x. We obtain an expression for the variance by 
starting from (7.4.2) and observing that the terms in (7.4.2) are all independent from 
each other: atxo is a constant and the disturbances £, are ii.d. by assumption. The 
variance of x; is therefore given by the sum of variances (compare (7.3.2) for the general 
case including the covariance), 


_ ot 
Var (x4) = 0 + Xi (ae); Var (e;) = 07s" (ay aoe =, (a?) = oP -5 f 
(7.4.5) 


We see that it is also a function of t. The fact that, for 0 < a < 1, the variance becomes 
larger the higher t appears intuitively clear. The further we look into the future, the “more 
randomness” there is: equation (7.4.2) shows that a higher t means that more random 
variables are added up. (One should keep in mind, however, that i.i.d. variables are added 
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up. If they were negatively correlated, the variance would not necessarily increase over 
time.) 

The fact that we used Var (atzo) = 0 here also shows that we work with a conditional 
distribution of x+. We base our computation of the variance at some future point in time 
on our knowledge of the RV X in t = 0. If we wanted to make this point very clearly, we 
could write Varo (x+). 


e Long-run behaviour 


When we considered deterministic difference equations like (2.5.6), we found the long- 
run behaviour of x; by computing the solution of this difference equation and by letting 
t approach infinity. This would give us the fixpoint of this difference equation as e.g. in 
(2.5.7). The concept that corresponds to a fixpoint/ steady state in an uncertain envi- 
ronment, e.g. when looking at stochastic difference equations like (7.4.1), is the limiting 
distribution. As stated earlier, stochastic difference equations do not tell us anything 
about the evolution of x; itself, they only tell us something about the distribution of x. 
It would therefore make no sense to ask what 2; is in the long run - it will always remain 
random. It makes a lot of sense, however, to ask what the distribution of x; is for the 
long run, i.e. for t — oo. 

All results so far were obtained without a specific distributional assumption for €: 
apart from specifying a mean and a variance. Understanding the long-run distribution of 
x, in (7.4.1) is easy if we assume that the stochastic component £; is normally distributed 
for each t, e ~ N (1,07). In this case, starting in t = 0, the variable xı is from (7.4.1) 
normally distributed as well. As a weighted sum of two random variables that are (jointly) 
normally distributed gives again a random variable with a normal distribution (where the 
mean and variance can be computed as in ch. 7.3.2), we also know from (7.4.1) that z1, 
LQ... Lz, ... are all normally distributed. 

As the normal distribution is a two-parameter distribution characterized by the mean 
and variance, we can find a stable distribution of x; for the long-run if the mean and 
variance approach some constant. Neglecting the cases of a > 1, the mean č of our 
long-run normal distribution is given from (7.4.4) by the fixpoint 


lim % = č% =~, 0<a<l. 
t—0o l-a 


The variance of the long-run distribution is from (7.4.5) 


1 
25 2 
Jim var (x4) = 0 To 


Hence, the long-run distribution of x+ for 0 < a < 1 is a normal distribution with mean 
/ (1 — a) and variance o?/ (1 — a’). 


7.4. Examples of stochastic difference equations 171 


e The evolution of the distribution of x; 


Given our results on the evolution of the mean and the variance of x, and the fact 
that we assumed ¢; to be normally distributed, we know that x; is normally distributed at 
each point in time. Hence, in order to understand the evolution of the distribution of x, 
we only have to find out how the expected value and variance of the variable x; evolves. 
As we have computed just this above, we can express the density for x; in closed form by 
(compare ch. 7.2.2) 

1 i (2) ž 
e 2 Ct 


ie) == 
V 2r0? 
where the mean u, and the variance o? are functions of time. These functions are given 


by u, = č, from (7.4.2) and o? = Var (x+) from (7.4.5). For some illustrating parameter 
values, we can then draw the evolution of the distribution of x as in the following figure. 


A simple stochastic difference equation 
Xt = 44-1 + Er poe 


initial distribution 


limiting distribution 


Figure 7.4.1 Evolution of a distribution over time 


Remember that we were able to plot a distribution for each t only because of properties 
of the normal distribution. If we had assumed that £+ is lognormally or equally distributed, 
we would not have been able to say something about the distribution of x, easily. The 
means and variances in (7.4.4) and (7.4.5) would still have been valid but the distribution 
of x, for future t is generally unknown for distributions of e+ other than the normal 
distribution. 
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e An interpretation for many agents 


Let us now make an excursion into the world of heterogeneous agents. Imagine the 
variable xj, represents the financial wealth of an individual i. Equation (7.4.1) can then 
be interpreted to describe the evolution of wealth x; over time, 


Lit = ALit—1 + Eit- 


Given information in period t = 0, it predicts the density function f(x) for wealth in 
all future periods, given some initial value xj9. Shocks are identically and independently 
distributed (i.i.d.) over time and across individuals. 

Assuming a large number of agents 7, more precisely a continuum of agents with mass 
n, a law of large numbers can be applied: Let all agents start with the same wealth xj) = £o 
int = 0. Then the density f (x4) for the wealth of any individual i in t equals the realized 
wealth distribution in t for the continuum of agents with mass n. Put differently: when 
the probability for an individual to hold wealth between some lower and upper bound in 
t is given by p, the share of individuals that hold wealth between these two bounds in t is 
also given by p. Total wealth in t in such a pure idiosyncratic risk setup (i.e. no aggregate 
uncertainty) is then deterministic (as are all other population shares) and is given by 


C= nf Tit f (ade = pls (7.4.6) 


OO 


where u, is the average wealth over individuals or the expected wealth of any individual 
i. 

The same argument can also be made with a discrete number of agents. The wealth 
xi, of individual ¿ at t is a random variable. Let the probability that zi is smaller than 
T be given by P (xa < z) = p. Now assume there are n agents and therefore, at each 
point in time t, there are n independent random variables z;. Denote by ñ the number of 
random variables x; that have a realization smaller than z. The share of random variables 
that have a realization smaller than Z is denoted by q = 7/n. It is then easy to show 
that Eq = p for all n and, more importantly, lim,_... var (q) = 0. This equals in words 
the statement based on Judd above: The share in the total population n is equal to the 
individual probability if the population is large. This share becomes deterministic for 
large populations. 

Here is now a formal proof: Define Y; as the number of realizations below z for one 
Tin Le. Y; = I (xu < 7) where J is the indicator function which is 1 if the condition in 
parentheses holds and 0 if not. Clearly, the probability that Y; = 1 is given by p, i.e. 
Y; is Bernoulli(p) distributed, with EY; = p and varY; = p(1 — p). (The Y; are iid. if 
the x, are i.i.d..) Then the share g is given by q = %_,Y;/n. Its moments are Eg = p 
and varg = p(1 — p)/n. Hence, the variance tends to zero for n going to infinity. (More 
technically, g tends to p in “quadratic mean” and therefore “in probability”. The latter 
means we have a “weak law of large numbers” .) 
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7.4.2 A more general case 
Let us now consider the more general difference equation 
Ly = Lei + Et, 
where the coefficient a is now also stochastic 
on (4,02), &~ (4,02). 


Analyzing the properties of this process is pretty complex. What can be easily done, 
however, is to analyze the evolution of moments. As Vervaat (1979) has shown, the 
limiting distribution has the following moments 


Bg =o) Bate) Ba". 
Assuming we are interested in the expected value, we would obtain 
Bx! = 5l (1) E (ate!) Ex* = (1) E (0%!) Ex? + (2) E (a'e®) Ba! 
= Ee + Bak. 


Solving for Ex yields Ex = ~*~ =;t. 
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Chapter 8 


Two-period models 


8.1 An overlapping generations model 


Let us now return to maximization issues. We do so in the context of the most sim- 
ple example of a dynamic stochastic general equilibrium model. It is a straightforward 
extension of the deterministic model analyzed in section 2.4. 

The structure of the maximization problem of individuals, the timing of when uncer- 
tainty reveals itself and what is uncertain depends on the fundamental and exogenous 
sources of uncertainty. As the fundamental source of uncertainty results here from the 
technology used by firms, technologies will be presented first. Once this is done, we can 
derive the properties of the maximization problem households or firms face. 


8.1.1 Technology 


Let there be an aggregate technology 
Y; = AKE LI” (8.1.1) 


The total factor productivity level A; is uncertain. We assume that A; is identically and 
independently distributed (i.i.d.) in each period. The random variable A; is positive, 
has a mean A and a variance o”. No further information is needed about its distribution 
function at this point, 

A~ (A,o7), A4 > 0. (8.1.2) 
Assuming that total factor productivity is i.i.d. means, inter alia, that there is no tech- 
nological progress. One can imagine many different distributions for A;. In principle, all 
distributions presented in the last section are viable candidates. Hence, we can work with 
discrete distributions or continuous distributions. 


8.1.2 Timing 


The sequence of events is as follows. At the beginning of period t, the capital stock 
K; is inherited from the last period, given decisions from the last period. Then, total 
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factor productivity is revealed. With this knowledge, firms choose factor employment and 
households choose consumption (and thereby savings). 


eo ^N e 


Figure 8.1.1 Timing of events 


Hence, only “at the end of the day” does one really know how much one has produced. 
This implies that wages and interest payments are also only known with certainty at the 
end of the period. 

The state of the economy in t is completely described by K,, L; and the realization of 
A. All variables of the model are contingent on the state. 


8.1.3 Firms 


As a consequence of this timing of events, firms do not bear any risk and they pay the 
marginal product of labour and capital to workers and capital owners at the end of the 
period, 


OY; 
sip 8.1.3 
Wt Par, ( ) 
OY; AN 
Be E Lg Aa C , 8.1.4 
NRM A a(%) (Saa) 


All risk is therefore born by households through labour and capital income. 
In what follows, the price will be set to unity, p = 1. All other prices will therefore 
be real prices in units of the consumption good. 


8.1.4 Intertemporal utility maximization 


This is the first time in this book that we encounter a maximization problem with un- 
certainty. The presentation will therefore be relatively detailed in order to stress crucial 
features which are new due to the uncertainty. 


e General approach 


We consider an agent that lives for two periods and works and consumes in a world as 
just described. Agents consume in both periods and choose consumption such that they 
maximize expected utility. In all generality concerning the uncertainty, she maximizes 


max E; {u (c+) + Bu (c41) }, (8.1.5) 


8.1. An overlapping generations model 177 


where Ø is the subjective discount factor that measures the agent’s impatience to con- 
sume. The expectations operator has an index t, similar to Eo in (7.4.3), to indicate that 
expectations are based on the knowledge concerning random variables which is available 
in period t. We will see in an instant whether the expectations operator E, is put at 
a meaningful place in this objective function. Placing it in front of both instantaneous 
consumption from c; and from cı is the most general way of handling it. We will also 
have to specify later what the control variable of the household is. 

Imagine the agent chooses consumption for the first and the second period. When 
consumption is chosen and given wage income w;, savings adjust such that the budget 
constraint 

We = Ce + St (8.1.6) 


for the first period holds. Note that this constraint always holds, despite the uncertainty 
concerning the wage. It holds in realizations, not in expected terms. In the second 
period, the household receives interest payments on savings made in the first period and 
uses savings plus interests for consumption, 


(1 + 441) St = Ct+1- (8.1.7) 


One way of solving this problem (for an alternative, see the exercise) is to insert 
consumption levels from these two constraints into the objective function (8.1.5). This 
gives 

max E, {u (we — s+) + Bu((1 + ress) s+)} - 


This nicely shows that the household influences consumption in both periods by choosing 
savings s+ in the first period. In fact, the only control variable the household can choose 
1S Sz. 

Let us now take into consideration, as drawn in the above figure, that consumption 
takes place at the end of the period after revelation of productivity A; in that period. 
Hence, the consumption level in the first period is determined by savings only and is 
thereby certain. Note that even if consumption c; (or savings) was chosen before reve- 
lation of total factor productivity, households would want to consume a different level 
of consumption c; after A; is known. The first choice would therefore be irrelevant and 
we can therefore focus on consumption choice after revelation of uncertainty right away. 
The consumption level in the second period is definitely uncertain, however, as the next 
period interest rate rı depends on the realization of A;,, which is unknown in t when 
decisions about savings s; are made. The objective function can therefore be rewritten as 


max u (wi — st) + BEru ((1 + rii) se). 


For illustration purposes, let us now assume a discrete random variable A, with a finite 
number n of possible realizations. Then, this maximization problem can be written as 


max {u (we — st) + BXi_y meu ((1 + rit) st)} 


178 Chapter 8. Two-period models 


where 7; is the probability that the interest rate r; 441 is in state 7 in +1. This is the same 
probability as the probability that the underlying source of uncertainty A; is in state 7. 
The first-order condition then reads 


w (we — St) = BUF miu’ (1+ Titti) St) (1+ rieti) = Eru (1 + rest) se) (1 + rea) 

(8.1.8) 
Marginal utility of consumption today must equal expected marginal utility of consump- 
tion tomorrow corrected by interest and time preference rate. Optimal behaviour in an 
uncertain world therefore means ex ante optimal behaviour, i.e. before random events 
are revealed. Ex post, i.e. after resolution of uncertainty, behaviour is suboptimal when 
compared to the case where the realization is known in advance: Marginal utility in ¢ will 
(with high probability) not equal marginal utility (corrected by interest and time prefer- 
ence rate) in t+ 1. This reflects a simple fact of life: “If I had known before what would 
happen, I would have behaved differently.” Ex ante, behaviour is optimal, ex post, prob- 
ably not. Clearly, if there was only one realization for A;, i.e. mı = 1 and7;=0Vi>1, 
we would have the deterministic first-order condition we had in exercise 1 in ch. 2. 

The first-order condition also shows that closed-form solutions are possible if marginal 
utility is of a multiplicative type. As savings s; are known, the only quantity which is 
uncertain is the interest rate r,,1. If the instantaneous utility function u (.) allows us to 
separate the interest rate from savings, i.e. the argument (1 + rj441) sin wu’ ((1 + fit+) Sz) 
in (8.1.8), an explicit expression for s; and thereby consumption can be computed. This 
will be shown in the following example and in exercise 6. 


e An example - Cobb-Douglas preferences 


Now assume the household maximizes a Cobb-Douglas utility function as in (2.2.1). In 
contrast to the deterministic setup in (2.2.1), however, expectations about consumption 
levels need to be formed. Preferences are therefore captured by 


E: {yln c + (1 —y) ln c1}. (8.1.9) 


When we express consumption by savings, we can express the maximization problem by 
maxs, Es {y In (wi — st) + (1 — y) In ((1 + ri+1) 5¢)} which is identical to 


max yln(w; — s+) + (1 — y) [ln s; + Ey ln (1 + ria). (8.1.10) 


The first-order condition with respect to savings reads 


io 
a (8.1.11) 
Wt — St St 


and the optimal consumption and saving levels are given by the closed-form solution 


G= 7, C= 1-yA+reiw, s=1-y)w, (8.1.12) 
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just as in the deterministic case (2.4.7). 


Thus, despite the setup with uncertainty, one can compute a closed-form solution of 
the same structure as in the deterministic solutions (2.2.4) and (2.2.5). What is peculiar 
here about the solutions and also about the first-order condition is the fact that the ex- 
pectations operator is no longer visible. One could get the impression that households 
do not form expectations when computing optimal consumption paths. The expectations 
operator “got lost” in the first-order condition only because of the logarithm, i.e. the 
Cobb-Douglas nature of preferences. Nevertheless, there is still uncertainty for an indi- 
vidual being in t : consumption in t+ 1 is unknown as it is a function of the interest-rate 
int+1. 

Exercise 6 shows that closed-form solutions are possible also for the CRRA case beyond 
Cobb-Douglas. 


8.1.5 Aggregation and the reduced form for the CD case 


We now aggregate over all individuals. Let there be L newborns each period. Consump- 
tion of all young individuals in period t is given from (8.1.12), 


CH = Lyw = (1 — a). 


The second equality used competitive wage setting from (8.1.3), the fact that with a Cobb- 
Douglas technology (8.1.3) can be written as wL; = p: (1 — a) Y;, the normalization of 
pı to unity and the identity between number of workers and number of young, L; = L. 
Note that this expression would hold identically in a deterministic model. With (8.1.4), 
consumption by old individuals amounts to 


C? = Lily [1 + r] w = (1-91 +r) — 0) Yi. 


Consumption in t depends on output in t — 1 as savings are based on income in t — 1. 

The capital stock in period t+ 1 is given by savings of the young. One could show this 
as we did in the deterministic case in ch. 2.4.2. In fact, adding uncertainty would change 
nothing to the fundamental relationships. We can therefore directly write 


Kia = Ls; = L[L—7]w; = (1-9) 1-0) % = (1-9) (1-0) AKEL, 


where we used the expression for savings for the Cobb-Douglas utility case from (8.1.12). 
Again, we succeeded in reducing the presentation of the model to one equation in one 
unknown. This allows us to illustrate the dynamics of the capital stock in this economy 
by using a simple phase diagram. 
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Ki = Kin 


Kə = (1 = y)(1 Te a)A Kel? 


Kipi 
K3 = (1 = y)(1 = a)A K9 L!” 
Kı = (1 = y)(1 = a)Apk¢L-* 


Ko Kı Ko K3 Ky 


Figure 8.1.2 Convergence towards a “stochastic steady state” 


In principle this phase diagram looks identical to the deterministic case. There is of 
course a fundamental difference. The K;,, loci are at a different point in each period. In 
t= 0, Ko and Ap are known. Hence, by looking at the Ky loci, we can compute Ky as in 
a deterministic setup. Then, however, TFP changes and, in the example plotted above, 
A, is larger than Ag. Once A, is revealed in t = 1, the new capital stock for period 2 
can be graphically derived. With A» revealed in t = 2, K; can be derived and so on. It 
is clear that this economy will never end up in one steady state, as in the deterministic 
case, as A; is different for each t. As illustrated before in fig. 7.4.1, however, the economy 
can converge to a unique stationary distribution for K4. 


8.1.6 Some analytical results 
e The basic difference equation 


In order to better understand the evolution of this economy, let us now look at some 
analytical results. The logarithm of the capital stock evolves according to 


ln Kia = ln (1—7) (1 — a)) + ln A4 + an K; + (1 — a)ln L. 


Assuming a constant population size L; = L, we can rewrite this equation as 


kipi = Mo tak, Hun vi ~ N(u,,02), (8.1.13) 

where we used 
kEhnK (8.1.14) 
mo =ln|(1-7) (0-a) + (1-—a)ln £ (8.1.15) 


and v; = ln A; captures the uncertainty stemming from the TFP level A;. As TFP is 
i.i.d., so is its logarithm v;. Since we assumed the TFP to be lognormally distributed, 
its logarithm v+ is normally distributed. When we remove the mean from the random 
variable by replacing according to vg = €+ + u, where e ~ N(0,02), we obtain k1 = 
Mo + Hy + QK + Et. 
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e The expected value and the variance 


We can now compute the expected value of k; by following the same steps as ch. 7.4.1, 
noting that the only additional parameter is mo. Starting in t = 0 with an initial value 
ko, and solving this equation recursively gives 

1=o! tintis 


ki = ako + (Mo + u) ea EXA Es. (8.1.16) 


Computing the expected value from the perspective of t = 0 gives 


1 — at 
Le’ 


Eok = atko + (Mo + u) (8.1.17) 
where we used Eoc, = 0 for all s > 0 (and we set £o = 0 or assumed that expectations in 
0 are formed before the realization of £o is known). For t going to infinity, we obtain 


hie =e 


8.1.18 
lim i=. (8.1.18) 


Note that the solution in (8.1.16) is neither difference- nor trend stationary. Only in the 
limit, we have a (pure) random walk. For a very large t, a‘ in (8.1.16) is zero and (8.1.16) 
implies a random walk, 


Kt — Ke-1 = E¢-1- 


The variance of k, can be computed with (8.1.16), where again independence of all 
terms is used as in (7.4.5), 


= 2t 
Var (k) =0+0+ Var (Sijat te) = Hihat? = - "o (8.1.19) 
—a 
In the limit, we obtain 
2 
lim Var (Kt) = = (8.1.20) 


A graphical illustration of these findings would look similar to the one in fig. 7.4.1. 
e Relation to fundamental uncertainty 


For a more convincing economic interpretation of the mean and the variance, it is 
useful to express some of the equations, not as a function of properties of the log of Aj, 
i.e. properties of 14, but directly of the level of A;. As (7.3.4) implies, the mean and 
variances of these two variables are related in the following fashion 


ii (: ce (2y) ; (8.1.21) 
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1 1 f 
Hy = npg — 50) = Mba — 5 in (: | @ | (8.1.22) 


HA 


We can insert these expressions into (8.1.17) and (8.1.19) and study the evolution over 
time or directly focus on the long-run distribution of k; = In K;. Doing so by inserting 


into (8.1.18) and (8.1.20) gives 
2 2 
mot mpa- $n (1+ (4) in (1+ (¢2)'] 
lim Var (k) = 


l-a t—00 1 — Q2 


lim Eo Kt = 
too 


These equations tell us that uncertainty in TFP as captured by o,4 not only affects 
the spread of the long-run distribution of the capital stock but also its mean. More 
uncertainty leads to a lower long-run mean of the capital stock. This is interesting given 
the standard result of precautionary saving and the portfolio effect (in setups with more 
than one asset). 


8.1.7 CRRA and CARA utility functions 


Before concluding this first analysis of optimal behaviour in an uncertain world, it is 
useful to explicitly introduce CRRA (constant relative risk aversion) and CARA (constant 
absolute risk aversion) utility functions. Both are widely used in various applications. 
The Arrow-Pratt measure of absolute risk aversion is —u” (c) /u’(c) and the measure of 
relative risk aversion is —cu” (c) /u’(c). An individual with uncertain consumption at a 
small risk would be willing to give up a certain absolute amount of consumption which 
is proportional to —u” (c) /u’ (c) to obtain certain consumption. The relative amount she 
would be willing to give up is proportional to the measure of relative risk aversion. 

The CRRA utility function is the same function as the CES utility function which we 
know from deterministic setups in (2.2.10) and (5.3.2). Inserting a CES utility function 
(ct — 1) / (1 — o) into these two measures of risk aversion gives a measure of absolute 


peT) 
elr) T 
is minus the inverse of the intertemporal elasticity of substitution. This is why the CES 


utility function is also called CRRA utility function. Even though this is not consistently 
done in the literature, it seems more appropriate to use the term CRRA (or CARA) in 
setups with uncertainty only. In a certain world without risk, risk-aversion plays no role. 

The fact that the same parameter o captures risk aversion and intertemporal elasticity 
of substitution is not always desirable as two different concepts should be captured by 
different parameters. The latter can be achieved by using a recursive utility function of 
the Epstein-Zin type. 

The typical example for a utility function with constant absolute risk aversion is the 
exponential utility function u (c(r)) = —e™7° where o is the measure of absolute risk 
aversion. Given relatively constant risk-premia over time, the CRRA utility function 
seems to be preferable for applications. 

See “further reading” on references to a more in-depth analysis of these issues. 


risk aversion of — = g /c (T) and a measure of relative risk aversion of ø, which 
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8.2 Risk-averse and risk-neutral households 


Previous analyses in this book have worked with the representative-agent assumption. 
This means that we neglected all potential differences across individuals and assumed 
that they are all the same (especially identical preferences, labour income and initial 
wealth). We now extend the analysis of the last section by (i) allowing individuals to give 
loans to each other. These loans are given at a riskless endogenous interest rate r. As 
before, there is a “normal” asset which pays an uncertain return r;,1. We also (ii) assume 
there are two types of individuals, the risk-neutral ones and the risk-averse, denoted by 
i =a,n. The second assumption is crucial: if individuals were identical, i.e. if they had 
identical preferences and experienced the same income streams, no loans would be given 
in equilibrium. In this world with heterogeneous agents, we want to understand who owns 
which assets. We keep the analysis simple by analyzing a partial equilibrium setup. 


e Households 


The budget constraints (8.1.6) and (8.1.7) of all individuals 7 now read 
w= Ct si, (8.2.1) 


Gamsi iar] (8.2.2) 


In the first period, there is the classic consumption-savings choice. In addition, there 
is an investment problem as savings need to be allocated to the two types of assets. 
Consumption in the second period is paid for entirely by capital income. Interests paid 
on the portfolio amount to 

r = Pry + (1-6) r. (8.2.3) 


Here and in subsequent chapters, 6’ denotes the share of wealth held by individual 7 in 
the risky asset. 

We solve this problem by the substitution method which gives us an unconstrained 
maximization problem. A household 7 with time preference rate p and therefore discount 
factor 6 = 1/ (1 + p) maximizes 


Uj = u (u — si) + Eru ([1 + r*] si) > max (8.2.4) 


i pi 
84,0 


by now choosing two control variables: The amount of resources not used in the first 
period for consumption, i.e. savings s$, and the share 0" of savings held in the risky asset. 
First-order conditions for si and 6", respectively, are 


w (4) = bE; {u (Gai) [+r], (8.2.5) 


Bul (41) [re — r] = 0. (8.2.6) 
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Note that the first-order condition for consumption (8.2.5) has the same interpretation, 
once slightly rewritten, as the interpretation in deterministic two-period or infinite horizon 
models (see (2.2.6) in ch. 2.2.2 and (3.1.6) in ch. 3.1.2). When rewriting it as 


E; k (i) 1 | zi (8.2.7) 


aeS TET) 


we see that optimal behaviour again requires us to equate marginal utilities today and 
tomorrow (the latter in its present value) with relative prices today and tomorrow (the 
latter also in its present value). Of course, in this stochastic environment, we need to 
express everything in expected terms. As the interest rates and consumption tomorrow 
are jointly uncertain, we can not bring it exactly in the form as known from above in 
(2.2.6) and (3.1.6). However, this will be possible, further below in (9.1.10) in ch. 9.1.3. 

The first-order condition for 0 says that expected returns from giving a loan and 
holding the risky asset must be identical. Returns consist of the interest rate times 
marginal utility. This condition can best be understood when first thinking of a certain 
environment. In this case, (8.2.6) would read rı = r : agents would be indifferent 
when holding two assets only if they receive the same interest rate on both assets. Under 
uncertainty and with risk-neutrality of agents, i.e. u’ = const., we get Eyrz4, =r : Agents 
hold both assets only if the expected return from the risky assets equals the certain return 
from the riskless asset. 

Under risk-aversion, we can write this condition as Eu’ (ci, 1) re = Eyu’ (ci44) r (or, 
given that r is non-stochastic, as Fru’ G a) Traa = Tha G ae, This says that agents 
do not value the interest rate per se but rather the extra utility gained from holding an 
asset: An asset provides interest of 7:4; which increases utility by u’ (a fc rt41. The share 
of wealth held in the risky asset then depends on the increase in utility from realizations 
of 7441 across various states and the expectations operator computes a weighted sum of 
theses utility increases: Epu’ (Cua) Ta = YiU (Eaa) Tjt+1Tj, Where j is the state, fjt+1 
the interest rate in this state and 7, the probability for state j to occur. An agent is 
then indifferent between holding two assets when expected utility-weighted returns are 
identical. 


e Risk-neutral and risk-averse behaviour 


For risk-neutral individuals (i.e. the utility function is linear in consumption), the 
first-order conditions become 


1 = BE [L+ Oraa + (1-8) ri), (8.2.8) 


Er =r. (8.2.9) 


The first-order condition for how to invest implies, together with (8.2.8), that the endoge- 
nous interest rate for loans is pinned down by the time preference rate, 


1=8jl+r]er=p (8.2.10) 
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as the discount factor is given by 6 = (1+ p)~' as stated before in (8.2.4). Reinserting this 
result into (8.2.9) shows that we need to assume that an interior solution for Eri, = p 
exists. This is not obvious for an exogenously given distribution for the interest rate rı 
but it is more plausible for a situation where r; is stochastic but endogenous as in the 
analysis of the OLG model in the previous chapter. 
For risk-averse individuals with u (c?) = In cf, the first-order condition for consumption 

reads with 7 in (8.2.3) being replaced by a 

1 1 1 1 

— = BE [1+ 0ra + (1 —-8") r] = BE= = B= (8.2.11) 
t 


c Chia St si 


where we used (8.2.2) for the second equality and the fact that s; as a control variable is 
deterministic for the third. Hence, as in the last section, we can derive explicit expressions. 
Use (8.2.11) and (8.2.1) and find 


a a a a a 1 
si = ba & we — cf = Pa ecg = T+ pu 
This gives with (8.2.1) again and with (8.2.2) 
a p a p a a 
st [1 + 0ra + (1 — 8%) r] w. (8.2.12) 


14h" MH IFA 


This is our closed-form solution for risk-averse individuals in our heterogeneous-agent 
economy. 

Let us now look at the investment problem of risk-averse households. The derivative 
of their objective function is given by the left-hand side of the first-order condition (8.2.6) 
times 6. Expressed for logarithmic utility function, and inserting the optimal consumption 
result (8.2.12) yields 


d Tt Zr 
—U* = BE — -r]=(1+8)E 
dé U; p Cy [ri r] ( B) 1 + OTi + (1 = 0°) r 
Tiy >T X 
= (14 E =(14 E i 8.2.13 


The last step defined X = rı — r as a RV and ¢ as a constant. 
e Who owns what? 


It can now easily be shown that (8.2.13) implies that risk-averse individuals will not 
allocate any of their savings to the risky asset, i.e. 0° = 0. First, observe that the 
derivative of the expression ÆX/ (ġ + 0° X) from (8.2.13) with respect to 0° is negative 


d X X? 
—E—— = E | -—]<0 v. 
d0% ¢ + 6°X ( a 
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The sign of this derivative can also easily be seen from (8.2.13) as an increase in 6“ implies 
a larger denominator. Hence, when plotted, the first-order condition is downward sloping 
in 0°. Second, by guessing, we find that, with (8.2.9), 0° = 0 satisfies the first-order 
condition for investment, 


X 
C=) SH = EX =0. 
o+0°X 


Hence, the first-order condition is zero for 0° = 0. Finally, as the first-order condition is 
monotonically decreasing, 0° = 0 is the only value for which it is zero, 


X 


E =0S67 =0. 
fae > O 


This is illustrated in the following figure. 


Figure 8.2.1 The first-order condition (8.2.13) for the share 0° of savings held in the 
risky asset 


The figure shows that expected utility of a risk-averse individual increases for negative 
0° and falls for positive 0°. Risk-averse individuals will hence allocate all of their savings 
to loans, ie. 6° = 0. They give loans to risk-neutral individuals who in turn pay a 
certain interest rate equal to the expected interest rate. All risk is born by risk-neutral 
individuals. 


8.3. Pricing of contingent claims and assets 187 
8.3 Pricing of contingent claims and assets 


8.3.1 The value of an asset 


The question is what an individual is willing to pay in t = 0 for an asset that is worth pj 
(which is uncertain) in t = 1. As the assumption underlying the pricing of assets is that 
individuals behave optimally, the answer is found by considering a maximization problem 
where individuals solve a saving and portfolio problem. 


e The individual 
Let an individual’s preferences be given by 


E {u (co) + Bu(cr)}. 


This individual can invest in a risky and in a riskless investment form. It earns labour 
income w in the first period and does not have any other source of income. This income 
w is used for consumption and savings. Savings are allocated to several risky assets j and 
a riskless bond. The first period budget constraint therefore reads 


w = Co + UY _1MjP50 + b 


where the number of shares j are denoted by m,, their price by pjo and the amount of 
wealth held in the riskless asset is b. The individual consumes all income in the second 
period which implies the budget constraint 


Cy = Li_ymypji + (1 +1) 6, 


where r is the interest rate on the riskless bond. 
We can summarize this maximization problem as 
ae (co) + BEu ((1 +r) (w — co — Lymypjo) + XLymypj1) « 
Lii 

This expression illustrates that consumption in period one is given by capital income 
from the riskless interest rate r plus income from assets j. It also shows that there is 
uncertainty only with respect to period one consumption. This is because consumption 
in period zero is a control variable. The consumption in period one is uncertain as prices 
of assets are uncertain. In addition to consumption co, the number m; of shares bought 
for each asset are also control variables. 


e First-order conditions and asset pricing 


Computing first-order conditions gives 


u' (co) = (1 +r) BEu' (c1) 
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This is the standard first-order condition for consumption. The first-order condition for 
the number of assets to buy reads for any asset 7 


BE [w (a) (1+ r) (—pjo) + pja)] = 0 E [w (c1) pa] = (1+ r) pjoEu' (c1). 


The last step used the fact that the interest rate and pjo are known in 0 and can therefore 
be pulled out of the expectations operator on the right-hand side. Reformulating this 
condition gives 
1 u (c 
= E ( 1) Pj 
l1+r Eu’ (ce) 


7 (8.3.1) 
which is an expression that can be given an interpretation as a pricing equation. The 
price of an asset j in period zero is given by the discounted expected price in period one. 
The price in period one is weighted by marginal utilities. 


8.3.2 The value of a contingent claim 


What we are now interested in are prices of contingent claims. Generally speaking, con- 
tingent claims are claims that can be made only if certain outcomes occur. The simplest 
example of the contingent claim is an option. An option gives the buyer the right to buy 
or sell an asset at a set price on or before a given date. We will consider current contingent 
claims whose values can be expressed as a function of the price of the underlying asset. 
Denote by g (p;1) the value of the claim as a function of the price p;ı of asset j in period 
one. The price of the claim in zero is denoted by v (p;o) . 

When we add an additional asset, i.e. the contingent claim under consideration, to 
the set of assets considered in the previous section, then optimal behaviour of households 
implies 

1 u’ (c1) 


v (Pjo) = i ca Eu (ey) 9 Pa) (8.3.2) 


If all agents were risk neutral, we would know that the price of such a claim would be 
given by 


v (Djo) = Tac (p51) (8.3.3) 


Both equations directly follow from the first-order condition for assets. Under risk neu- 
trality, a household’s utility function is linear in consumption and the first derivative 
is therefore a constant. Including the contingent claim in the set of assets available to 
households, exactly the same first-order condition for their pricing would hold. 


8.3.3 Risk-neutral valuation 


There is a strand in the finance literature, see e.g. Brennan (1979), that asks under 
what conditions a risk neutral valuation of contingent claims holds (risk neutral valuation 
relationship, RNVR) even when households are risk averse. This is identical to asking 
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under which conditions marginal utilities in (8.3.1) do not show up. We will now briefly 
illustrate this approach and drop the index j. 

Assume the distribution of the price pı can be characterized by a density f (pı, u), 
where u = Ep,;. Then, if a risk neutral valuation relationship exists, the price of the 
contingent claim in zero is given by 


1 


ET g (p1) f (pı; u) dpı, with p= (1+ r) po. 


v (po) = 
This is (8.3.3) with the expectations operator being replaced by the integral over realiza- 
tions g (pı) times the density f (pı). With risk averse households, the pricing relationship 
would read, under this distribution, 


v o) = ef Ea OD) F (pnn) dp: 


This is (8.3.2) expressed without the expectations operator. The expected value p is left 
unspecified here as it is not a priori clear whether this expected value equals (1 + r) po 
also under risk aversion. It is then easy to see that a RNVR. holds if u’ (c1) /Eu’ (c) = 
f (pı) /f (pı, p) . Similar conditions are derived in that paper for other distributions and 
for longer time horizons. 


8.4 Natural volatility I 


Natural volatility is a view of why growing economies experience phases of high and phases 
of low growth. The central belief is that both long-run growth and short-run fluctuations 
are jointly determined by economic forces that are inherent to any real world economy. 
Long-run growth and short-run fluctuations are both endogenous and two sides of the 
same coin: They both stem from the introduction of new technologies. 

It is important to note that no exogenous shocks occur according to this approach. 
In this sense, it differs from real business cycle (RBC) and sunspot models and also from 
endogenous growth models with exogenous disturbances. 

There are various models that analyse this view in more detail and an overview is 
provided at http://www.waelde.com/nv.html. This section will look at a simple model 
that provides the basic intuition. More on natural volatility will follow in ch. 11.5. 


8.4.1 The basic idea 


The basic mechanism of the natural volatility literature (and this is probably a necessary 
property of any model that wants to explain both short-run fluctuations and long-run 
growth) is that some measure of productivity (this could be labour or total factor produc- 
tivity) does not grow smoothly over time as in most models of exogenous or endogenous 
long-run growth but that productivity follows a step function. 
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Figure 8.4.1 Smooth productivity growth in balanced growth models (dashed line) and 
step-wise productivity growth in models of natural volatility 


With time on the horizontal and the log of productivity on the vertical axis, this figure 
shows a smooth productivity growth path as the dashed line. This is the smooth growth 
path that induces balanced growth. In models of natural volatility, the growth path of 
productivity has periods of no change at all and points in time of discrete jumps. After a 
discrete jump, returns on investment go up and an upward jump in growth rates results. 
Growth rates gradually fall over time as long as productivity remains constant. With the 
next jump, growth rates jump up again. While this step function implies long-run growth 
as productivity on average grows over time, it also implies short-run fluctuations. 

The precise economic reasons given for this step function - which is not simply imposed 
but always follows from some deeper mechanisms - differ from one approach to the other. 
A crucial implication of this step-function is the implicit belief that economically relevant 
technological jumps take place once every 4-5 years. Each cycle of an economy and 
therefore also long-run growth go back to relatively rare events. Fluctuations in time 
series that are of higher frequency than these 4-5 years either go back to exogenous 
shocks, to measurement error or other disturbances in the economy. 

The step function sometimes captures jumps in total factor productivity, sometimes 
only in labour productivity for the most recent vintage of a technology. This difference 
is important for the economic plausibility of the models. Clearly, one should not build a 
theory on large aggregate shocks to TFP, as those are not easily observable. Some papers 
in the literature show indeed how small changes in technology can have large effects (see 
“further reading” ). 

This section presents the simple stochastic natural volatility model which allows us to 
show the difference from exogenous shock models in the business cycle most easily . 


8.4.2 A simple stochastic model 


This section presents the simplest possible model that allows us to understand the differ- 
ence between the stochastic natural volatility and the RBC approach. 
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e Technologies 
Let the technology be described by a Cobb-Douglas specification, 
Y, = AK, 


where A; represents total factor productivity, K+ is the capital stock and L are hours 
worked. As variations in hours worked are not required for the main argument here, we 
consider L constant. Capital can be accumulated according to 


Kı = (1 are ô) K+ h 


Total factor productivity follows 


Aisi = (1 + dt) At (8.4.1) 
where 
oe og : “ae Pt 
q = l 0 \ with probability l ne (8.4.2) 
The probability depends on resources R, invested into R&D, 
pr = p (Re). (8.4.3) 


Clearly, the function p (R+) in this discrete time setup must be such that 0 < p (R) < 1. 
The specification of technological progress in (8.4.2) is probably best suited to point 
out the differences to RBC type approaches: The probability that a new technology occurs 
is endogenous. This shows both the “new growth literature” tradition and the differences 
from endogenous growth type RBC models. In the latter approach, the growth rate is 
endogenous but shocks are still exogenously imposed. Here, the source of growth and 
fluctuations all stem from one and the same source, the jumps in q in (8.4.2). 


e Optimal behaviour by households 
The resource constraint the economy needs to obey in each period is given by 
Ci + h+ R= Y, 


where C;, J; and R; are aggregate consumption, investment and R&D expenditure, respec- 
tively. Assume that optimal behaviour by households implies consumption and investment 
into R&D amounting to 

C: = sc Yı, Ri = SRY+, 


where sc is the consumption and spr the saving rate in R&D. Both of them are con- 
stant. This would be the outcome of a two-period maximization problem or an infinite 
horizon maximization problem with some parameter restriction. As the natural volatility 
literature has various papers where the saving rate is not constant, it seems reasonable 
not to develop an optimal saving approach here fully as it is not central to the natural 
volatility view. See “further reading”, however, for references to papers which develop a 
full intertemporal approach. 
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8.4.3 Equilibrium 


Equilibrium is determined by 


Koai = (1-8) K +Y, — R- C, 
= (1— ô) Ki + (1 — sr — so) Y. 


plus the random realization of technology jumps, where the probability of a jump depends 
on investment in R&D, 
Pi = P (SRY:). 

Assume we start with a technological level of Aj. Let there be no technological progress 
for a while, i.e. gq, = 0 for a certain number of periods t. Then the capital stock converges 
to its “temporary steady state” defined by Kı = K= K, 
| eee 

ot Ae Ai. (8.4.4) 

ô 
In a steady state, all variables are constant over time. Here, variables are constant only 
temporarily until the next technology jump occurs. This is why the steady state is said 
to be temporary. The convergence behaviour to the temporary steady state is illustrated 
in the following figure. 


ôK = (1 ESRT sc) Ao KL!” < Ke = 


K, Ki,7 =(1-8) K; + 1-Sp-Sc)A12K PL" 


+1 + 


K, 


+ 


j=(1-8)K; H1-Sr-S) AKL” 


Ko K, 
Figure 8.4.2 The Sisyphus economy - convergence to a temporary steady state 


With a new technology coming in, say, period 12, the total factor productivity in- 
creases, according to (8.4.1), from Ao to A2 = (1 + q) Ao. The K:+ı line shifts upwards, 
as shown in fig. 8.4.2. As a consequence, which can also be seen in (8.4.4), the steady 
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state increases. Subsequently, the economy approaches this new steady state. As growth 
is higher immediately after a new technology is introduced (growth is high when the econ- 
omy is far away from its steady state), the growth rate after the introduction of a new 
technology is high and then gradually falls. Cyclical growth is therefore characterized by 
a Sisyphus-type behaviour: K, permanently approaches the current, temporary steady 
state. Every now and then, however, this steady state jumps outwards and capital starts 
approaching it again. 


8.5 Further reading and exercises 


More on basic concepts of random variables than in ch. 7.1 and 7.2 can be found in Evans, 
Hastings and Peacock (2000). A very useful reference is Spanos (1999) who also treats 
functions of several random variables or Severini (2005). For a more advanced treatment, 
see Johnson, Kotz and Balakrishnan (1995). The results on distributions here are used 
and extended in Bossmann, Kleiber and Walde (2007). 

There is a long discussion on the application of laws of large numbers in economics. 
An early contribution is by Judd (1985). See Kleiber and Kotz (2003) for more on 
distributions. 

The example for the value of an asset is inspired by Brennan (1979). 

The literature on natural volatility can be found in ch. 11 on p. 293. 

See any good textbook on Micro or Public economics (e.g. Mas-Colell et al., 1995, 
Atkinson and Stiglitz, 1980) for a more detailed treatment of measures of risk aversion. 
There are many papers using a CARA utility function. Examples include Hassler et al. 
(2005), Acemoglu and Shimer (1999) and Shimer and Werning (2007, 2008). 

Epstein-Zin preferences have been developed by Epstein and Zin (1989) for discrete 
time setups and applied inter alia in Epstein and Zin (1991). For an application in 
macroeconomics in discrete time building on the Kreps-Porteus approach, see Weil (1990). 
The continuous time representation was developed in Svensson (1989). See also Obstfeld 
(1994) or Epaulard and Pommeret (2003). 
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Exercises Chapter 7 and 8 
Applied Intertemporal Optimization 


Stochastic difference equations and 
applications 


1. Properties of random variables 
Is the following function a density function? Draw this function. 


flx) =a, A>0. (8.5.1) 
State possible assumptions about the range of x and values for a. 


2. The exponential distribution 
Assume the time between two rare events (e.g. the time between two earthquakes 
or two bankruptcies of a firm) is exponentially distributed. 
(a) What is the distribution function of an exponential distribution? 
(b) What is the probability of no event between 0 and z? 
(c) What is the probability of at least one event between 0 and x? 


3. The properties of uncertain technological change 
Assume that total factor productivity in (8.1.2) is given by 


(a) 


(b) the density function E 
f(A), At E [A,A], 


(c) the probability function g(A;), A; € {41, Ao, ..., An}, 
(d) A; = A; with probability p; and 


(e) In Arı = yln Ay + E441. Make assumptions for y and ¢,;; and discuss them. 


What is the expected total factor productivity and what is its variance? What is 
the expected output level and what is its variance, given a technology as in (8.1.1)? 
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4. Stochastic difference equations 
Consider the following stochastic difference equation 


Yt+1 = byt +m + Ut+1, Vg ~ N (0, o?) 


(a) Describe the limiting distribution of y+. 


(b) Does the expected value of y; converge to its fixpoint monotonically? How does 
the variance of y; evolve over time? 


5. Saving under uncertainty 
Consider an individual’s maximization problem 
max Fs {u (cr) + Bu (c1) } 
subject to w; = Ce + 5, (1+ ri) 5¢ = Ciy 


(a) Solve this problem by replacing her second period consumption by an expres- 
sion that depends on first period consumption. 


(b) Consider now the individual’s decision problem given the utility function u (c+) = 
c7. Should you assume a parameter restriction on o? 


(c) What is your implicit assumption about 8? Can it be negative or larger than 
one? Can the time preference rate p, where 8 = (1 + o! , be negative? 


6. Closed-form solution for a CRRA utility function 
Let households maximize a CRRA-utility function 
U: = Ei [y +l — near] = ve? + 9) Bet 
subject to budget constraints (8.1.6) and (8.1.7). 
(a) Show that an optimal consumption-saving decision given budget constraints 


(8.1.6) and (8.1.7) implies savings of 
Wt 


eee (ee es i 
1+ {a} 


where ¢ = 1/0 is the intertemporal elasticity of substitution and ® = F; K + meae] 
is the expected (transformed) interest rate. Show further that consumption 
when old is 


St 


Gai = +r) z 
1+4 2 \ 


and that consumption of the young is 


(i—-7)® 


Ct = ice (8.5.2) 


y 6. 
l f} 
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(b) Discuss the link between the savings expression s; and the one for the logarith- 
mic case in (8.1.12). Point out why c+; is uncertain from the perspective of t. 
Is c uncertain from the perspective of t - and of t — 1? 


7. OLG in general equilibrium 
Build an OLG model in general equilibrium with capital accumulation and auto- 
regressive total factor productivity, In Aşı = yIn Ay + E441 with e441 ~ N (ce, 07). 
What is the reduced form? Can a phase-diagram be drawn? 


8. Asset pricing 
Under what conditions is there a risk neutral valuation formula for assets? In other 
words, under which conditions does the following equation hold? 
1 


o = — E : 
Pj l+r Pj 


Chapter 9 


Multi-period models 


Uncertainty in dynamic models is probably most often used in discrete time models. 
Looking at time with a discrete perspective has the advantage that timing issues are very 
intuitive: Something happens today, something tomorrow, something the day before. 
Time in the real world, however, is continuous. Take any two points in time, you will 
always find a point in time in between. What is more, working with continuous time mod- 
els under uncertainty has quite some analytical advantages which make certain insights - 
after an initial heavier investment into techniques - much simpler. We will nevertheless 
follow the previous structure of this book and first present models in discrete time. 


9.1 Intertemporal utility maximization 


Maximization problems in discrete times can be solved by using many methods. One 
particularly useful one is - again - dynamic programming. We therefore start with this 
method and consider the stochastic sibling to the deterministic case in ch. 3.3. 


9.1.1 The setup with a general budget constraint 


Due to uncertainty, our objective function is slightly changed. In contrast to (3.1.1), it 
now reads 


U, = hy 8 u (c+), (9.1.1) 


where the only difference lies in the expectations operator E;. As we are in an environment 
with uncertainty, we do not know all consumption levels c, with certainty. We therefore 
need to form expectations about the implied instantaneous utility from consumption, 
u (c+). The budget constraint is given by 


Tt+1 = f (Tt, Ct, Et) : (9.1.2) 


This constraint shows why we need to form expectations about future utility: The value 
of the state variable x7;,,; depends on some random source, denoted by e+. Think of this 
c, as uncertain TFP or uncertain returns on investment. 
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9.1.2 Solving by dynamic programming 
The optimal program is defined in analogy to (3.3.2) by 
V (a1, €4) = mak U, subject to £ty1 = f (Lt, Ct, €t). 
The additional term in the value function is ¢;. It is useful to treat £; explicitly as a state 
variable for reasons we will soon see. Note that this issue is related to the discussion of 


“what is a state variable?” in ch. 3.4.2. In order to solve the maximization problem, we 
again follow the three step scheme. 


e DP1: Bellman equation and first-order conditions 
The Bellman equation is 


V (Tr, Et) = max {u (cz) + BEV (41, Et+1)} $ 


It again exploits the fact that the objective function (9.1.1) is additively separable, de- 
spite the expectations operator, assumes optimal behaviour as of tomorrow and shifts the 
expectations operator behind instantaneous utility of today as u (c+) is certain given that 
G is a control variable. The first-order condition is 


Of (.) 
Oct 


OFt41 — o0. 
oc 

(9.1.3) 
It corresponds to the first-order condition (3.3.5) in a deterministic setup. The “only” 
difference lies in the expectations operator E. Marginal utility from consumption does not 
equal the loss in overall utility due to less wealth but the expected loss in overall utility. 
Equation (9.1.3) provides again an (implicit) functional relationship between consumption 
and the state variable, c, = c (x1, €+). As we know all state variables in t, we know the 
optimal choice of our control variable. Be aware that the expectations operator applies 
to all terms in the brackets. If no confusion arises, these brackets will be omitted in what 
follows. 


u (c) + BE; Vrei (e441 Et+1) =u (&) + BE; Verp Siena) 


e DP2: Evolution of the costate variable 


The second step under uncertainty also starts from the maximized Bellman equation. 
The derivative of the maximized Bellman equation with respect to the state variable is 
(using the envelope theorem) 


OTt 


ox; 


Observe that 7,41 by the constraint (9.1.2) is given by f (£1, Ct, €+) , ie. by quantities that 
are known in t. Hence, the derivative 0x;41/0r; = fr, is non-stochastic and we can write 
this expression as (note the similarity to (3.3.6)) 


Vo, (£t, Et) = Byala (Beer Et+1). (9.1.4) 


Voy (£i, Et) = BE Voss (Titi, €t41) 
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This step clearly shows why it is useful to include €; as a state variable into the arguments 
of the value function. If we had not done so, one could get the impression, for the same 
argument that x;,, is known in t due to (9.1.2), that the shadow price V,, (£t+1, 141) is 
non-stochastic as well and the expectations operator would not be needed. Given that the 
value of the optimal program depends on £+, however, it is clear that the costate variable 
V’ (t41,€t41) is random in t indeed. (Some of the subsequent applications will treat €, 
as an implicit state variable and will not always be as explicit as here.) 


e DP3: Inserting first-order conditions 


Given that Of (.) /Oc = fe is non-random in t, we can rewrite the first-order con- 
dition as wu’ (ct) + bfe EtV (t41, €t41) = 0. Inserting it into (9.1.4) gives Vr, (£t, €t) = 


— Bey! (c+) . Shifting this expression by one period yields V,,,,, (t141, 141) = — fersa Ww (C41). 
ct Ct4+1 
Inserting Vz, (£+, €+) and V2,,, (£t+1, Et+1) into the costate equation (9.1.4) again, we obtain 
fe 
u' (ct) = BE faust (Cr41) : 
Ct+1 


9.1.3 The setup with a household budget constraint 


Let us now look at a first example - a household that maximizes utility. The objective 
function is given by (9.1.1), 
U; = E&L, 8u (c). 


It is maximized subject to the following budget constraint (of which we know from (2.5.13) 
or (3.6.6) that it fits well into a general equilibrium setup), 


At41 = (1 + rt) Qt + We — Ptt. (9.1.5) 


Note that the budget constraint must hold after realization of random variables, not in 
expected terms. From the perspective of t, all prices (r;, w+, p+) in t are known, prices in 
t+ 1 are uncertain. 


9.1.4 Solving by dynamic programming 


e DP1: Bellman equation and first-order conditions 


Having understood in the previous general chapter that uncertainty can be explicitly 
treated in the form of a state variable, we limit our attention to the endogenous state 
variable a; here. It will turn out that this keeps notation simpler. The value of optimal 
behaviour is therefore expressed by V (a+) and the Bellman equation can be written as 


V (a) = max {u (cr) + BEV (at41)}. 
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The first-order condition for consumption is 


Oat 


u (c) + BEV" (ar41) = u (c) — BEV" (at+1) Pe = 0, 


where the second step computed the derivative by using the budget constraint (9.1.5). 
Rewriting the first-order condition yields 


u (c4) = BpiEiV” (ai41) (9.1.6) 
as the price p; is known in t. 
e DP2: Evolution of the costate variable 


Differentiating the maximized Bellman equation gives (using the envelope theorem) 
V’ (a) = BEV" (at41) Oa¢41/Oa,;. Again using the budget constraint (9.1.5) for the partial 
derivative, we find 


V’ (a) = B [L + ri] EV” (aes). (9.1.7) 


Again, the term [1 + r;| was put in front of the expectations operator as r; is known 
in t. This difference equation describes the evolution of the shadow price of wealth in the 
case of optimal consumption choices. 


e DP3: Inserting first-order conditions 


Inserting the first-order condition (9.1.6) gives V’ (as) = [1+ r4] st, Inserting this 


t 
expression into the differentiated maximized Bellman equation (9.1.7) twice gives a nice 


Euler equation, 


A = B [1 + ri] Er [L + re 


w (C1) res (ce) Bul (cr41) 
t z 
Pt Pt+H1 Pt (1+ raa) Pipi 


[1 + r] 


« (9.18) 


Rewriting it as we did before with (8.2.7), we get 


Bu (C41) Pt \ 
E, : a4 9.1.9 
í w (c) (14r) Piri l ) 


which allows us to give the same interpretation as the first-order condition in a two-period 
model, both deterministic (as in eq. (2.2.6) in ch. 2.2.2) and stochastic (as in eq. (8.2.5) 
in ch. 8.2) and as in deterministic infinite horizon models (as in eq. (3.1.6) in ch. 3.1.2): 
Relative marginal utility must be equal to relative marginal prices - taking into account 
that marginal utility in t + 1 is discounted at 8 and the price in t + 1 is discounted by 
using the interest rate. 

Using two further assumptions, the expression in (9.1.8) can be rewritten such that 
we come even closer to the deterministic Euler equations: First, let us choose the output 
good as numéraire and thereby set prices p; = pı = p as constant. This allows us to 
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remove p; and pı from (9.1.8). Second, we assume that the interest rate is known (say 
we are in a small open economy with international capital flows). Hence, expectations are 
formed only with respect to consumption in t+ 1. Taking these two aspects into account, 
we can write (9.1.8) as 
w (ea) 1 
BE (ca) 1/ (1+ rey) 


This is now as close to (2.2.6) and (3.1.6) as possible. The ratio of (expected discounted) 
marginal utilities is identical to the ratio of relative prices. 


(9.1.10) 


9.2 A central planner 


Let us now consider the classic central planner problem for a stochastic growth economy. 
We specify an optimal growth model where total factor productivity is uncertain. In 
addition to this, we also allow for oil as an input good for production. This allows us to 
understand how some variables can easily be substituted out in a maximization problem 
even though we are in an intertemporal stochastic world. 

Consider a technology where output is produced with oil O; in addition to the standard 
factors of production, 

Y, = AKPOL °? 


Again, total factor productivity A; is stochastic. Now, the price of oil, q@, is stochastic as 
well. Let capital evolve according to 


Kipi = (1 = ô) Ky + Y; = qO = Ci (9.2.1) 


which is a trade balance and good market clearing condition all in one. The central 
planner maximizes 

max BXL, Btu (C, 

EO) t ae ( ) 
by choosing a path of aggregate consumption flows C, and oil consumption O,. At t, all 
variables indexed t are known. The only uncertainty concerns TFP and the price of oil 
in future periods. 


e DP1: Bellman equation and first-order conditions 


The Bellman equation reads V (Ky) = maxc,o, {u (Ci) + BEV (Ki+1)}. The only 
state variable included as argument is the capital stock. Other state variables (like the 
price q, of oil) could be included but would not help in the derivation of optimality 
conditions. See the discussion in ch. 3.4.2. The first-order condition for consumption is 


dV (Keys) 
dK 41 


ul (C,) + BE, [-1] =0 4 w' (C,) = BE,V’ (Kis) 
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For oil it reads 
o OY; 
EV (Kea) — IY: — Od = 0S —— = q. 

BEV (Kea) 50, (Y: — O] 50, 7% 
The last step used the fact that all variables in t are known and the partial derivative with 
respect to O; can therefore be moved in front of the expectations operator - which then 
cancels. We therefore have obtained a standard period-for-period optimality condition as 
it is known from static problems. This is the typical result for a control variable which 
has no intertemporal effects as, here, imports of oil affect output Y; and the costs qO; in 
the resource constraint (9.2.1) contemporaneously. 


e DP2: Evolution of the costate variable 


The derivative of the Bellman equation with respect to the capital stock K, gives 
(using the envelope theorem) 


ð OY; ; 
V’ (K) = Prag Y (Ki41) = B h — ô+ ae EV (Ki+1) (9.2.2) 


just as in the economy without oil. The term E — ô+ va can be pulled out of the ex- 


pectation operator as A, and thereby pa is known at the moment of the savings decision. 
e DP3: Inserting first-order conditions 


Following the same steps as above without oil, we again end up with 


u (Cy) = pEr (Cipa) h — ô+ | 


OK 


The crucial difference is, that now expectations are formed with respect to technological 
uncertainty and uncertainty concerning the price of oil. 


9.3 Asset pricing in a one-asset economy 


This section returns to the question of asset pricing. Ch. 8.3.1 treated this issue in 
a partial equilibrium setting. Here, we take a general equilibrium approach and use a 
simple stochastic model with one asset, physical capital. We then derive an equation that 
expresses the price of capital in terms of income streams from holding capital. In order 
to be as explicit as possible about the nature of this (real) capital price, we do not choose 
a numéraire good in this section. 
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9.3.1 The model 


e Technologies 


The technology used to produce a homogeneous output good is of the simple Cobb- 

Douglas form 
Y, = AKEL”, (9.3.1) 

where TFP 4; is stochastic. Labour supply L is exogenous and fix, the capital stock is 
denoted by K+. 

e Households 

Household preferences are standard and given by 

U, = E&L, Bn (c). 


We start from a budget constraint that can be derived like the budget constraint 
(2.5.10) in a deterministic world. Wealth is held in units of capital K, where the price of 
one unit is v4. When we define the interest rate as 


K 
ao. (9.3.2) 
Ut 
the budget constraint (2.5.10) reads 
kent = (L+ re) ke + = = a (9.3.3) 
t t 


e Goods market 


Investment and consumption goods are traded on the same goods market. Total supply 
is given by Y;, demand is given by gross investment Kı — K;+6K; and consumption C4. 
Expressed in a well-known way, goods market equilibrium yields the resource constraint 
of the economy, 

Kai = K+ Y¥, -—C, — 6K}. (9.3.4) 


9.3.2 Optimal behaviour 
Firms maximize instantaneous profits which implies first-order conditions 


Factor rewards are given by their value marginal products. 

Given the households’ preferences and the constraint in (9.3.3), optimal behaviour by 
households is described by (this follows identical steps as for example in ch. 9.1.3 and is 
treated in ex. 2), 

u' (C4) =) Bul (Cii) 

Pi/ Vt Gh +01) Pes /Ye 
This is the standard Euler equation extended for prices, given that we have not chosen a 
numéraire. We replaced c; by C; to indicate that this is the evolution of aggregate (and 
not individual) consumption. 


(9.3.6) 
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9.3.3 The pricing relationship 


Let us now turn to the main objective of this section and derive an expression for the 
real price of one unit of capital, i.e. the price of capital in units of the consumption good. 
Starting from the Euler equation (9.3.6), we insert the interest rate (9.3.2) in its general 
formulation, i.e. including all prices, and rearrange to find 


K 
1 Ut 1 Wii Ut+1 1 Ut+1 OYi41 
Ci) — = BE,u (C 14 -ô | — = BB (C 1 — ô) — ‘ 
ul (C) = BBa (Con) (14 SE — 5) 2 = pew Crs) (1-8) 4 mk: | 
9.3.7 


Now define a discount factor ®,,; = u’ (C1) /u’ (C;) and d; as the “net dividend pay- 
ments”, i.e. payments to the owner of one unit of capital. Net dividend payments per 
unit of capital amount to the marginal product of capital OY,/0K, minus the share 6 of 
capital that depreciates - that goes kaput - each period times the real price v;/p; of one 


unit of capital, di = a — ô a Inserting this yields 
v v 
— = bE fan + re . (9.3.8) 
Pt Pt+1 


Note that all variables uncertain from the perspective of today in t appear behind the 
expectations operator. 

Now assume for a second that we are in a deterministic world and the economy is 
in a steady state. Equation (9.3.8) could then be written with ®,,; = 1 and without 


Vt41 
Pt+1 
forward, starting in v, and inserting repeatedly gives 


the expectations operator as p =p dey + | . Solving this linear differential equation 


v B Ce +8 Ce +8 Ce +8 faus j mall 


Pe t+4 


v 
= Bdisi + B’dira + B?dits + B diga + a 
t+4 


Continuing to insert, one eventually and obviously ends up with 
Ut 


Vet 
eS Ei l dis + Be 2 
Pt Pt+T 


The price v; of a unit of capital is equal to the discounted sum of future dividend payments 

plus its discounted price (once sold) in t+ T. In an infinite horizon perspective, this 
becomes 7 j 

2 = EL, B*diys + lim BT. 

Pt Toco PtT 

In our stochastic setup, we can proceed according to the same principles as in the 

deterministic world but need to take the expectations operator and the discount factor 


®, into account. We replace ea in (9.3.8) by BE 1in2 [dez + oee | and then v442/pi+2 


Pt+2 
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and so on to find 


z v 
~ = GED Ce + BEty1Pi+2 | diye + OEP Ce pa ee] | 


Pt Pt+3 
Ut 
= E; ce Te BPD, Des odes + B’ Dii Pier Pps 3dirg T Porbiabea] 
t+3 
>P Aag EN, (9.3.9) 


Pt+3 
where we defined the discount factor to be 


ul (Citn) u (Cits) 
A, = BIR bun = CIE _, ———._ = 2° ——. 
p n=1 *t+ p n=1 u! (Cx) p F (Ca) 

The discount factor adjusts discounting by the preference parameter 6, by relative mar- 
ginal consumption and by prices. Obviously, (9.3.9) implies for larger time horizons 
n= PXT Asdis + EiArvr. Again, with an infinite horizon, this reads 

v 

= = Fyre Asdi+s + jm EATV T. (9.3.10) 

Pt oo 
The real price v;/p,; amounts to the discounted sum of future dividend payments d;,,. The 
discount factor is A, which contains marginal utilities, relative prices and the individual’s 
discount factor 8. The term limy_,., EtArvr is a “bubble term” for the price of capital 
and can usually be set equal to zero. As the derivation has shown, the expression for the 
price v;/p, is “simply” a rewritten version of the Euler equation. 


9.3.4 More real results 


e The price of capital again 


The result on the determinants of the price of capital is useful for economic intuitions 
and received a lot of attention in the literature. But can we say more about the real price 
of capital? The answer is yes and it comes from the resource constraint (9.3.4). This 
constraint can be understood as a goods market clearing condition. The supply of goods 
Y, equals demand resulting from gross investment Kı — K: + ôK, and consumption. 
The price of one unit of the capital good therefore equals the price of one unit of the 
consumption and output good, provided that investment takes place, i.e. J; > 0. Hence, 


Ut = Pr.. (9.3.11) 


The real price of capital v;/p; is just equal to one. Not surprisingly, capital goods and 
consumption goods are traded on the same market. 
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e The evolution of consumption (and capital) 


When we want to understand what this model tells us about the evolution of consump- 
tion, we can look at a modified version of (9.3.6) by inserting the interest rate (9.3.2) with 
the marginal product of capital from (9.3.5) and the price expression (9.3.11), 


u (Ci) = BE’ (Cr41) (1 + OK == 5) 7 


This is the standard Euler equation (see e.g. (9.1.10)) that predicts how real consumption 
evolves over time, given the real interest rate and the discount factor 8. 

Together with (9.3.4), we have a system in two equations that determine C; and K; 
(given appropriate boundary conditions). The price p, and thereby the value v, can not 
be determined (which is of course a consequence of Walras’ law). The relative price is 
trivially unity from (9.3.11), v,/p; = 1. Hence, the predictions concerning real variables 
do not change when a numéraire good is not chosen. 


e An endowment economy 


Many papers work with pure endowment economies. We will look at such an economy 
here and see how this can be linked to our setup with capital accumulation. Consider 
an individual that can save in one asset and whose budget constraint is given by (9.3.3). 
Let this household behave optimally such that optimal consumption follows (9.3.7). Now 
change the capital accumulation equation (9.3.4) such that - for whatever reasons - K is 
constant and let also, for simplicity, depreciation be zero, 6 = 0. Then, output is given 
according to (9.3.1) by Y, = A,K°L*~°, i.e. it follows some exogenous stochastic process, 
depending on the realization of A;. This is the exogenous endowment of the economy for 
each period t. Further, consumption equals output in each period, C; = Y;. 

Inserting output into the Euler equation (9.3.7) gives 


OY, 
ul (4) = BBW! (Wis) (0-0) SE + TE) 
Pt Pt+1 t+1 


The equation shows that in an endowment economy where consumption is exogenously 
given at each point in time and households save by holding capital (which is constant on 
the aggregate level), the price v;/p; of the asset changes over time such that households 
want to consume optimally the exogenously given amount Y;. This equation provides a 
description of the evolution of the price of the asset in an endowment economy. These 
aspects were analyzed for example by Lucas (1978) and many others. 


9.4. Endogenous labour supply 207 


9.4 Endogenous labour supply 


e The maximization problem 


The analysis of business cycles is traditionally performed by including an endogenous 
labour supply decision in the consumption and saving framework we know from ch. 9.1. 
We will now solve such an extended maximization problem. 

The objective function (9.1.1) is extended to reflect that households value leisure. We 
also allow for an increase in the size of the household. The objective function now reads 


U, = EL, p u (en T — lr) nr, 


where T is the total endowment of this individual with time and l, is hours worked in 
period 7. Total time endowment T is, say, 24 hours or, subtracting time for sleeping 
and other regular non-work and non-leisure activities, 15 or 16 hours. Consumption per 
member of the family is given by c, and u (.) is instantaneous utility of this member. The 
number of members in 7 is given by n,. 

The budget constraint of the household is given by 


Qtyy = (1 H ri) Ôi + ralew, — nce 


where â, = naş is household wealth in period t. Letting w; denote the hourly wage, n4l,w; 
stands for total labour income of the family, i.e. the product of individual income lyw 
times the number of family members. Family consumption in t is nc. 


e Solving by the Lagrangian 


We now solve the maximization problem max U; subject to the constraint by choosing 
individual consumption c, and individual labour supply l}. We solve this by using a 
Lagrange function. A solution by dynamic programming would of course also work. 

For the Lagrangian we use a Lagrange multiplier A, for the constraint in 7. This makes 
it different to the Lagrange approach in ch. 3.1.2 where the constraint was an intertemporal 
budget constraint. It is similar to ch. 3.7 where an infinite number of multipliers is also 
used. In that chapter, uncertainty is, however, missing. The Lagrangian here reads 


L= ES, {87 *u (en, T — lr) nr + àr [(1 + r+) âr + nrlrWwr — nre — âr+1]} . 


We first compute the first-order conditions for consumption and hours worked for one 
point in time s, 


og u (Cs, T — ls) Nns — Ans} = (0; (9.4.1) 


Lia er" |- ITL)“ (cs, T — | Ns + Anas} =0. (9.4.2) 
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As discussed in 3.7, we also need to compute the derivative with respect to the state 
variable. Consistent with the approach in the deterministic setup of ch. 3.8, we compute 
the derivative with respect to the true state variable of the family, i.e. with respect to a,. 
This derivative is 


Lag = Ey{—As1+s[1+7,]} =0 
& Frs-1= E,{(1+ rs) As}. (9.4.3) 


This corresponds to (3.7.5) in the deterministic case, only that here we have an expecta- 
tions operator. 


e Optimal consumption 


As the choice for the consumption level c, in (9.4.1) will be made in s, we can assume 
that we have all information in s at our disposal. When we apply expectations E,, we see 
that all expectations are made with respect to variables of s. Cancelling n,, we therefore 
know that in all periods s we have 


cc, (cs, T — ls) = às. (9.4.4) 


s—t 
i OCs 


We can now replace Às and A,_; in (9.4.3) by the expressions we get from this equation 
where A, is directly available and A,_; is obtained by shifting the optimality condition 
back in time, i.e. by replacing s by s — 1. We then find 


ulema, T =i) = B Satr) pu (eT ~1)} 5 


Cs—1 S 


Bef wea T bay} 


Ob 554 


E, [a ð 


BE: fa +r) Žu (cs, T — i} 


S 


Now imagine, we are in s — 1. Then we form expectations given all the information we 
have in s — 1. As hours worked /,_; and consumption c,_; are control variables, they are 
known in s — 1. Hence, using an expectations operator E,—1, we can write 


Essi { 2 U (C54; T — ba) = 
OCs-1 


o 
u (cs-1, T = ls—1) = BE.-1 fa + Ts) dc u (cs, T = 1)} š 


Ob: T 


Economically speaking, optimal consumption-saving behaviour requires marginal utility 
from consumption by each family member in s — 1 to equal marginal utility in s, corrected 
for impatience and the interest rate. This condition is similar to the one in (9.1.8), only 
that here, we have utility from leisure as well. 
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e Labour-leisure choice 


Let us now look at the second first-order condition (9.4.2) to understand the condition 
for optimal intra-temporal labour supply. When an individual makes the decision of how 
much to work in s, she actually finds herself in s. Expectations are therefore formed in s 
and replacing t by s and rearranging slightly, the condition becomes, 


E, { (sao (c T- 1) m} LEAO: 


As none of the variables in the curly brackets are random from the perspective of s, 
after removing the expectations operator and cancelling n, on both sides, we obtain, 
ato" (cs, T — ls) = Asws. We finally use (9.4.4) expressed for t = s and with n, cancelled 
to obtain an expression for the shadow price As, Šu (c,, T — ls) = às. Inserting this yields 


Kara —I,) 
5 = Wy. 
aag t (Cs; T— Le) 


This is the standard condition known from static models. It also holds here as the 
labour-leisure choice is a decision made in each period and has no intertemporal dimension. 
The trade-off is entirely intra-temporal. In optimum it requires that the ratio of marginal 
utility of leisure to marginal utility of consumption is given by the ratio of the price of 
leisure - the wage w, - to the price of one unit of the consumption good - which is 1 here. 
Whether an increase in the price of leisure implies an increase in labour supply depends on 
properties of the instantaneous utility function u (.) . If the income effect caused by higher 
ws dominates the substitution effect, a higher wage would imply fewer hours worked. 
More on this can be found in many introductory textbooks to Microeconomics. 


9.5 Solving by substitution 


This section is on how to do without dynamic programming. We will get to know a method 
that allows us to solve stochastic intertemporal problems in discrete time without dynamic 
programming. Once this chapter is over, you will ask yourself why dynamic programming 
exists at all ... 


9.5.1 Intertemporal utility maximization 


The objective is again max,.,} Eo¥ Zobu (c+) subject to the constraint a441 = (1 + r+) at + 
w: — c¢. The household’s control variables are {c+} , the state variable is a, the interest 
rate r, and the wage w, are exogenously given. Now rewrite the objective function and 
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insert the constraint twice, 
Eo {Ebu (ci) + Bĉu (cs) + btu (Cs41) + D (cr) } 
= FoX obu (ce) + Eob u ((1+ rs)as + Ws — asy) 
+ Eq B°thu ((1 + rs41) @s41 + Ws+1 — a542) + Fou gy 98"u (cr) - 


Note that the expectations operator always refers to knowledge available at the beginning 
of the planning horizon, i.e. to t = 0. 

Now compute the first-order condition with respect to a,,,. This is unusual as we 
directly choose the state variable which is usually understood to be only indirectly influ- 
enced by the control variable. Clearly, however, this is just a convenient trick: by choosing 
ds41, Which is a state variable, we really choose c,. The derivative with respect to as+1 
yields 

EoB*u' (cs) = Eob ru! (Cs+1) Dar en), 


This almost looks like the standard optimal consumption rule. The difference lies in the 
expectations operator being present on both sides. This is not surprising as we optimally 
chose a;41 (i.e. cs), knowing only the state of the system in t = 0. 

If we now assume we are in s, our expectations would be based on knowledge in s and 
we could replace Ep by E,. We would then obtain EB u’ (cs) = 8*u’ (cs) for the left-hand 
side and our optimality rule reads 


u’ (cs) = BE su’ (C541) (1 + T 544i) 


This is the rule we know from Bellman approaches, provided e.g. in (9.1.8). 


9.5.2 Capital asset pricing 


Let us now ask how an asset that pays an uncertain return in T periods would be priced. 
Consider an economy with an asset that pays a return r in each period and one long- 
term asset which can be sold only after T periods at a price pr which is unknown today. 
Assuming that investors behave rationally, i.e. they maximize an intertemporal utility 
function subject to constraints, the price of the long-term asset can be found most easily 
by using a Lagrange approach or by straightforward inserting. 

We assume that an investor maximizes her expected utility Ey02,3"u (c+) subject to 
the constraints 


Co + Mopo + Ao = Wo, 
Ce +a, = (1+ r)a + w, 1<t<T-l, 
cr = Mopr + (1 +r)ar-ı + wr. 
In period zero, the individual uses labour income wọ to pay for consumption goods co, to 


buy mo units of the long-term asset and for “normal” assets ao. In periods one to T — 1, 
the individual uses her assets a;_; plus returns r on assets and her wage income w to 
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finance consumption and again buy assets a; (or keep those from the previous period). 
In the final period T, the long-term asset has a price of pr and is sold. Wealth from the 
previous period plus interest plus labour income wr are further sources of income to pay 
for consumption cr. Hence, the individual’s control variables are consumption levels c: 
and the number mo of long-term assets. 

This maximization problem can be solved most easily by inserting consumption levels 
for each period into the objective function. The objective function then reads 


u (wo — Mopo — ao) + Hoe btu (wi + (1+ r)a — a) 
+ EobTulmopr + (1 + rjar-ı + wr). 


What could now be called control variables are the wealth holdings a in periods t = 
0,..., T — 1 and (as in the original setup) the number of assets mo bought in period zero. 

Let us now look at the first-order conditions. The first-order condition for wealth in 
period zero is 


w (co) = (1 + r) BEou' (c1). (9.5.1) 
Wealth holdings in any period t > 0 are optimally chosen according to 
EoB'ul (ci) = Eob (1 + ryw (i41) & Eov' (ct) = 6 (1 + r) Bow! (cra). (9.5.2) 
We can insert (9.5.2) into the first-period condition (9.5.1) sufficiently often and find 
u' (co) = (1+ r}? 8° Eou (c2) =... = (1 + r)” 87 Eou (cr) (9.5.3) 
The first-order condition for the number of assets is 
pow! (co) = B* Epu'(cr)pr. (9.5.4) 


When we insert combined first-order conditions (9.5.3) for wealth holdings into the 
first-order condition (9.5.4) for assets, we obtain 


po(1 + r)” 87 Eou (er) = 8" Eou'(er)pr & 


/ 
peti ye 


Egu’ Gay 
which is an equation where we see the analogy to the two period example in ch. 8.3.1 
nicely. Instead of pjo = iE aay pjı in (8.3.1) where we discount by one period only 


and evaluate returns at expected marginal utility in period one, we discount by T periods 
and evaluate returns at marginal utility in period T. 

This equation also offers a lesson for life when we assume risk-neutrality for simplicity: 
if the payoff pr from a long-term asset is not high enough such that the current price is 
higher than the present value of the payoff, pp > (1 + r) T Eopr, then the long-term 
asset is simply dominated by short-term investments that pay a return of r per period. 
Optimal behaviour would imply not buying the long-term asset and just putting wealth 
into “normal” assets. This should be kept in mind the next time you talk to your insurance 
agent who tries to sell you life-insurance or private pension plans. Just ask for the present 
value of the payoffs and compare them to the present value of what you pay into the 
savings plan. 
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9.5.3 Sticky prices 
e The setup 


Sticky prices are a fact of life. In macroeconomic models, they are either assumed 
right away, or assumed following Calvo price setting or result from some adjustment-cost 
setup. Here is a simplified way to derive sluggish price adjustment based on adjustment 
cost. 

The firm’s objective function is to maximize its present value defined by the sum of 


discounted expected profits, 
1 Tt 
Vi = Ea ( ) Tr. 


l+r 


Profits at a point 7 > t in time are given by 

Tr = Prt — Wel, — © (pr, pr—1) 
where ® (p,, p;_1) are price adjustment costs. These are similar in spirit to the adjustment 
costs presented in ch. 5.5.1. We will later use a specification given by 


Q | Pr — Pr-1 i 
® (pr, Pr-1) = = p= = | 
Pr-1 


2 
This specification captures the essential mechanism that is required to make prices sticky, 
there are increasing costs in the difference p, — p,;_1. The fact that the price change is 
squared is not essential - in fact, as with all adjustment cost mechanisms, any power larger 
than 1 would do the job. More care about economic implications needs to be taken when 
a reasonable model is to be specified. 
The firm uses a technology 


(9.5.5) 


tr = Azle: 


We assume that there is a certain demand elasticity £ for the firm’s output. This can 
reflect a monopolistic competition setup. The firm can choose its output x, at each point 
in time freely by hiring the corresponding amount of labour /,. Labour productivity A, 
or other quantities can be uncertain. 


e Solving by substitution 


Inserting everything into the objective function yields 


1 Tt w, 
Vi = By, (=) (0-0, = at 8 a) 


Wp 


w 
= F (pe = T” — (Pnp) + F (vases Ta At — (Prap) 
t t+ 


1 T—t w, 
F Faye (=) (v-e, = Ant = a) : 
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The second and third line present a rewritten method which allows us to see the intertem- 
poral structure of the maximization problem better. We now maximize this objective 
function by choosing output x, for today. (Output levels in the future are chosen at a 
later stage.) 

The first-order condition is 


d[pit,] w, dọ (pi, P1) d® (Pi+1, Pt) 
E —— — — hk, = 0 S 
t ( dx, At dx, : dx, 
d [px] e Wi d® (pi, Pi—1) LE dÈ (pi41, Pe) 
dx, A; dx, i ; dx; l 
It has certain well-known components and some new ones. If there were no adjustment 
costs, i.e. ®(.) = 0, the intertemporal problem would become a static one and the 


usual condition would equate marginal revenue d |px] /dx; with marginal cost w;/At. 
With adjustment costs, however, a change in output today not only affects adjustment 
costs today Ps) but also (expected) adjustment costs py, Seem) tomorrow. As all 
variables with index t are assumed to be known in t, expectations are formed only with 
respect to adjustment costs tomorrow in t+ 1. 

Specifying the adjustment cost function as in (9.5.5) and computing marginal revenue 


using the demand elasticity €+ gives 


d |p] n Wi d | |P: — Pri Á LE d |P | Pii — Pe : i 
dx; A; i dx, |2 Pt-1 ome dz, | 2 Pt 
dp =i Wt Pt — Pt-1 1 dp, Pii — Pel d Piri 
— t t+ p = (ltée, ) p = o t £4 @ 
dx, f i ( : ) i Ar Pt-1 Pr_-1 di : Pt dx, pi 
— De 41- d 
B Wt $ É Pt | Pt er! zP fo [2 1 z Pm z) A 
Ar Pt-1 TPt- Pt pi dt 
(1 ae e7!) — = $ 2 = Pet Pt ey} <E fo e= y z| Bri Pt a) PA 
t Pt-1 TPt- Pt Pi Tt 
7 w — pe $ | eee — 
(1 eE +) Dt = a Q 2 = | re er — ——e, oF; een} 
t Pt-1 TPt- Pret Pt 
where €; = es is the demand elasticity for the firm’s good. Again, ¢ = 0 would give 


the standard static optimality condition (1 +E; 1) Pi = w;/A; where the price is a markup 
over marginal cost. With adjustment costs, prices change only slowly. 


9.5.4 Optimal employment with adjustment costs 
e The setup 


Consider a firm that maximizes profits as in ch. 5.5.1, 


1 
IL = Barang” (9.5.6) 
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We are today in t, time extends until infinity and the time between today and infinity 
is denoted by r. There is no particular reason why the planning horizon is infinity in 
contrast to ch. 5.5.1. Here we will stress that the optimality condition for employment is 
identical to a finite horizon problem. 

Instantaneous profits are given by the difference between revenue in t, which is iden- 
tical to output F (L) with an output price normalized to unity, labour cost w;L; and 
adjustment cost ® (L; — L4_1), 


TM = F (Li) — wili — (Lı a Lı) 3 (9.5.7) 


Costs induced by the adjustment of the number of employees between the previous period 
and today are captured by ®(.). Usually, one assumes costs both from hiring and from 
firing individuals, i.e. both for an increase in the labour force, L; — L-1 > 0, and from a 
decrease, L; — [4-1 < 0. A simple functional form for ®(.) which captures this idea is a 
quadratic form, i.e. ® (L, — L1) = ¢ (Lı — Day where ¢ is a constant. 

Uncertainty for a firm can come from many sources: Uncertain demand, uncertainty 
concerning the production process, uncertainty over labour costs or other sources. As 
we express profits in units of the output good, we assume that the real wage w, i.e. the 
amount of output goods to be paid to labour, is uncertain. Adjustment cost ® (L; — L4_1) 
are certain, i.e. the firm knows how many units of output profits reduce by when employ- 
ment changes by L — L1. 

As in static models of the firm, the control variable of the firm is employment Lz. 
In contrast to static models, however, employment decisions today in t not only affects 
employment today but also employment tomorrow as the employment decision in t affects 
adjustment costs in t + 1. There is therefore an intertemporal link the firm needs to take 
into account which is not present in the firm’s static models. 


e Solving by substitution 


This maximization problem can be solved directly by inserting (9.5.7) into the objec- 
tive function (9.5.6). One can then choose optimal employment for some point in time 
t < s < œ after having split the objective function into several subperiods - as for example 
in the previous chapter 9.5.1. The solution reads (to be shown in exercise 7) 


p (Liy <2 Li) 


F' (L) = ®' (L,—L,_,) —E 
(Lt) = w + © (Ly t-1) t [Ap 


When employment L; is chosen in t, there is only uncertainty concerning [,,1. The current 
wage w; (and all other deterministic quantities as well) are known with certainty. L;+1 is 
uncertain, however, from the perspective of today as the wage in 7 + 1 is unknown and 
L,41 will be have to be adjusted accordingly in 7 + 1. Hence, expectations apply only to 
the adjustment-cost term which refers to adjustment costs which occur in period t + 1. 
Economically speaking, given employment L;_; in the previous period, employment in t is 
chosen such that marginal productivity of labour equals labour costs adjusted for current 
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and expected future adjustment costs. Expected future adjustment costs are discounted 
by the interest rate r to obtain its present value. 

When we specify the adjustment cost function as a quadratic function, ® (L; — L-1) = 
g [Li — Lial , we obtain 


@ [Lin — L] | 


F' (L) = Li— Lal- E 
( t) w, + o [Lt gil t l4+r 


If there were no adjustment costs, i.e. ọ = 0, we would have F” (L;) = w;. Employment 
would be chosen such that marginal productivity equals the real wage. This confirms 
the initial statement that the intertemporal problem of the firm arises purely from the 
adjustment costs. Without adjustment costs, i.e. with ¢ = 0, the firm has the “standard” 
instantaneous, period-specific optimality condition. 


9.6 An explicit time path for a boundary condition 


Sometimes, an explicit time path for optimal behaviour is required. The transversality 
condition is then usually not very useful. A more pragmatic approach sets assets at some 
future point in time at some exogenous level. This allows us to then (at least numerically) 
compute the optimal path for all points in time before this final point easily. 

Let T be the final period of life in our model, i.e. set ar4, = 0 (or some other level 
for example the deterministic steady state level). Then, from the budget constraint, we 
can deduce consumption in T, 


ar+ı = (1 +rr)ar + wr- cr & cr =(l4+rr)art+ ur. 


Optimal consumption in T — 1 still needs to obey the Euler equation, compare for example 
to (9.1.9), i.e. 


u (er-1) = Er-ıß [1 + rr] u (er) : 


As the budget constraint requires 
ar = (1 +rr-1) ar-1 + wr-1 — cr-1, 
optimal consumption in T — 1 is determined by 


u (cr-1) = Erb [1 + rr] u (1 + rr) [1+ rr-1)ar-ı + wr-1 — cra] + wr) 


This is one equation in one unknown, cr-1, where expectations need to be formed 
about rr and wr and wr- are unknown. When we assume a probability distribution for 
rr and wr, we can replace Er- by a summation over states and solve this expression 
numerically in a straightforward way. 
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9.7 Further reading and exercises 


A recent introduction and detailed analysis of discrete time models with uncertainty in 
the real business cycle tradition with homogeneous and heterogenous agents is by Heer 
and Mausner (2005). Stokey and Lucas take a more rigorous approach to the one taken 
here (1989). An almost comprehensive in-depth presentation of macroeconomic aspects 
under uncertainty is provided by Ljungqvist and Sargent (2004). 

On capital asset pricing in one-sector economies, references include Jermann (1998), 
Danthine, Donaldson and Mehra (1992), Abel (1990), Rouwenhorst (1995), Stokey and 
Lucas (1989, ch. 16.2) and Lucas (1978). An overview is in ch. 13 of Ljungqvist and 
Sargent (2004). 

The example for sticky prices is inspired by Ireland (2004), going back to Rotemberg 
(1982). 

The statement that “the predictions concerning real variables do not change when a 
numéraire good is not chosen” is not as obvious as it might appear from remembering 
Walras’ law from undergraduate micro courses. There is a literature that analyses the 
effects of the choice of numéraire for real outcomes for the economy when there is imperfect 
competition. See e.g. Gabszewicz and Vial (1972) or Dierker and Grodahl (1995). 
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Exercises Chapter 9 
Applied Intertemporal Optimization 


Discrete time infinite horizon models under 
uncertainty 


1. Central planner 
Consider an economy where output is produced by 


Y; = AKEL!” 


Again, as in the OLG example in equation (8.1.1), total factor productivity A; is 
stochastic. Let capital evolve according to 


Kı = (1 — ô) Ki + Y; — C 
The central planner maximizes 


ESS Pulo 
mar E, 748" u (C) 


by again choosing a path of aggregate consumption flows C,. At t, all variables 
indexed ¢ are known. The only uncertainty concerns A1. What are the optimality 
conditions? 


2. A household maximization problem 
Consider the optimal saving problem of the household in ch. 9.3. Derive the Euler 
equation (9.3.6). 


3. Endogenous labour supply 
Solve the endogenous labour supply setup in ch. 9.4 by using dynamic programming. 


4. Closed-form solution 
Solve this model for the utility function u (C) = ca! and for 6 = 1. Solve it for 
a more general case (Benhabib and Rustichini, 1994). 


5. Habit formation 
Assume instantaneous utility depends, not only on current consumption, but also 
on habits (see for example Abel, 1990). Let the utility function therefore look like 


U; = FXL, uw (cr, vr), 
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where v, stands for habits like e.g. past consumption, v, = v (C;-1, Cr-2,...). Let 
such an individual maximize utility subject to the budget constraint 


Gig. = (1 + r4) ay + we — pec 


(a) Assume the individual lives in a deterministic world and derive a rule for an 
optimal consumption path where the effect of habits are explicitly taken into 
account. Specify habits by v, = c,_1. 


(b) Let there be uncertainty with respect to future prices. At a point in time t, 
all variables indexed by t are known. What is the optimal consumption rule 
when habits are treated in a parametric way? 


(c) Choose a plausible instantaneous utility function and discuss the implications 
for optimal consumption given habits v, = c,_1. 


6. Risk-neutral valuation 
Under which conditions is there a risk neutral valuation relationship for contingent 
claims in models with many periods? 


7. Labour demand under adjustment cost 
Solve the maximization problem of the firm in ch. 9.5.4 by directly inserting profits 
(9.5.7) into the objective function (9.5.6) and then choosing L+. 


8. Solving by substitution 
Solve the problem from ch. 9.5 in a slightly extended version, i.e. with prices p. 
Maximize EpX%,3"u (c+) by choosing a time path {c+} for consumption subject to 
Ary. = (14+ ri) ae + We — pice. 


9. Matching on labour markets 
Let employment L; in a firm follow 


Liya = (1 — 8) Li + pv, 


where s is a constant separation rate, u is a constant matching rate and V; denotes 
the number of jobs a firm currently offers. The firm’s profits m, in period 7 are 
given by the difference between revenue p,Y (L,) and costs, where costs stem from 
wage payments and costs for vacancies V, captured by a parameter y, 


Tr =prY (L,) —w,L, — yVz. 
The firm’s objective function is given by 
I = EEL, B T, 


where £ is a discount factor and E; is the expectations operator. 
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(a) Assume a deterministic world. Let the firm choose the number of vacancies 
optimally. Use a Lagrangian to derive the optimality condition. Assume that 
there is an interior solution. Why is this an assumption that might not always 
be satisfied from the perspective of a single firm? 


(b) Let us now assume that there is uncertain demand which translates into un- 
certain prices p, which are exogenous to the firm. Solve the optimal choice of 
the firm by inserting all equations into the objective function. Maximize by 
choosing the state variable and explain also in words what you do. Give an 
interpretation of the optimality condition. What does it imply for the optimal 
choice of V,? 


10. Optimal training for a marathon 

Imagine you want to participate in a marathon or any other sports event. It will 
take place in m days, i.e. in t+ m where t is today. You know that taking part in 
this event requires training e,, T € [t,t +m]. Unfortunately, you dislike training, 
i.e. your instantaneous utility u (e,) decreases in effort, u’ (e) < 0. On the other 
hand, training allows you to be successful in the marathon: more effort increases 
your personal fitness F,. Assume that fitness follows F,4, = (1 — ô) F, + e,, with 
0 <6 <1, and fitness at t + m is good for you yielding happiness of h (Fiim) . 


(a) Formally formulate an objective function which captures the trade-offs in such 
a training program. 


(b) Assume that everything is deterministic. How would your training schedule 
look (the optimal path of e,)? 


(c) In the real world, normal night life reduces fitness in a random way, i.e. 6 is 
stochastic. How does your training schedule look now? 
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Part IV 


Stochastic models in continuous time 
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Part IV is the final part of this book and, logically, analyzes continuous time models 
under uncertainty. The choice between working in discrete or continuous time is partly 
driven by previous choices: If the literature is mainly in discrete time, students will find 
it helpful to work in discrete time as well. The use of discrete time methods seem to hold 
for macroeconomics, at least when it comes to the analysis of business cycles. On the 
other hand, when we talk about economic growth, labour market analyses and finance, 
continuous time methods are very prominent. 


Whatever the tradition in the literature, continuous time models have the huge ad- 
vantage that they are analytically generally more tractable, once some initial investment 
into new methods has been digested. As an example, some papers in the literature have 
shown that continuous time models with uncertainty can be analyzed in simple phase 
diagrams as in deterministic continuous time setups. See ch. 10.6 and ch. 11.6 on further 
reading for references from many fields. 


To facilitate access to the magical world of continuous time uncertainty, part IV 
presents the tools required to work with uncertainty in continuous time models. It is 
probably the most innovative part of this book as many results from recent research flow 
directly into it. This part also most strongly incorporates the central philosophy behind 
writing this book: There will be hardly any discussion of formal mathematical aspects like 
probability spaces, measurability and the like. While some will argue that one can not 
work with continuous time uncertainty without having studied mathematics, this chapter 
and the many applications in the literature prove the opposite. The objective here is to 
clearly make the tools for continuous time uncertainty available in a language that is ac- 
cessible for anyone with an interest in these tools and some “feeling” for dynamic models 
and random variables. The chapters on further reading will provide links to the more 
mathematical literature. Maybe this is also a good point for the author of this book to 
thank all the mathematicians who helped him gain access to this magical world. I hope 
they will forgive me for “betraying their secrets” to those who, maybe in their view, were 
not appropriately initiated. 


Chapter 10 provides the background for optimization problems. As in part II where 
we first looked at differential equations before working with Hamiltonians, here we will 
first look at stochastic differential equations. After some basics, the most interesting 
aspect of working with uncertainty in continuous time follows: Ito’s lemma and, more 
generally, change-of-variable formulas for computing differentials will be presented. As 
an application of Ito’s lemma, we will get to know one of the most famous results in 
Economics - the Black-Scholes formula. This chapter also presents methods for how to 
solve stochastic differential equations or how to verify solutions and compute moments of 
random variables described by a stochastic process. 


Chapter 11 then looks once more at maximization problems. We will get to know 
the classic intertemporal utility maximization problem both for Poisson uncertainty and 
for Brownian motion. The chapter also shows the link between Poisson processes and 
matching models of the labour market. This is very useful for working with extensions 
of the simple matching model that allows for savings. Capital asset pricing and natural 
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volatility conclude the chapter. 


Chapter 10 


SDEs, differentials and moments 


When working in continuous time, uncertainty enters the economy usually in the form of 
Brownian motion, Poisson processes or Levy processes. This uncertainty is represented 
in economic models by stochastic differential equations (SDEs) which describe for exam- 
ple the evolution of prices or technology frontiers. This section will cover a wide range 
of differential equations (and show how to work with them) that appear in economics 
and finance. It will also show how to work with functions of stochastic variables, for 
example how output evolves given that TFP is stochastic or how wealth of a household 
grows over time, given that the price of the asset held by the household is random. The 
entire treatment here, as before in this book, will be non-rigorous and will focus on “how 
to compute things”. 


10.1 Stochastic differential equations (SDEs) 


10.1.1 Stochastic processes 


We got to know random variables in ch. 7.1. A random variable relates in some loose 
sense to a stochastic process of how (deterministic) static models relate to (deterministic) 
dynamic models: Static models describe one equilibrium, dynamic models describe a 
sequence of equilibria. A random variable has, “when looked at once” (e.g. when throwing 
a die once), one realization. A stochastic process describes a sequence of random variables 
and therefore, “when looked at once”, describes a sequence of realizations. More formally, 
we have the following: 


Definition 10.1.1 (Ross, 1996) A stochastic process is a parameterized collection of ran- 
dom variables 


{X (t) brettg, 7] - 


Let us look at an example for a stochastic process. We start from the normal distribu- 
tion of ch. 7.2.2 whose mean and variance are given by u and g? and its density function is 
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z> 


-1 
T= (v 20?) bales) Now define a normally distributed random variable Z (t) 


that has a variance that is a function of some t: instead of o°, write o7t. Hence, the 


-1 _ıfz- 2 
random variables we just defined have as density function f (z) = (v 20) e a(S ) : 


By having done so and by interpreting t as time, Z (t) is in fact a stochastic process: we 
have a collection of random variables, all normally distributed, they are parameterized by 
time t. 

Stochastic processes can be stationary, weakly stationary or non-stationary. Station- 
arity is a more restrictive concept than weak stationarity. 


Definition 10.1.2 (Ross, 1996, ch. 8.8): A process X (t) is stationary if X (t1),..., 
X (ta) and X (ty + 8),..., X (tn + 8$) have the same joint distribution for all n and s. 


An implication of this definition, which might help to get some “feeling” for this 
definition, is that a stationary process X (t) implies that, being in t = 0, X (tı) and 
X (t2) have the same distribution for all t > tı > 0. A weaker concept of stationarity only 
requires the first two moments of X (tı) and X (t2) (and a condition on the covariance) 
to be satisfied. 


Definition 10.1.3 (Ross, 1996) A process X (t) is weakly stationary if the first two mo- 
ments are the same for all t and the covariance between X (t2) and X (tı) depends only 
on tə =? th, 


EyX (t) =p, VarX(t)=07, Cov(X (t2), X (t1)) = f (te — tı), 
where u and o? are constants and f (.) is some function. 


Definition 10.1.4 A process which is neither stationary nor weakly stationary is non- 
stationary. 


Probably the best-known stochastic process in continuous time is the Brownian mo- 
tion. It is sometimes called the Wiener process after the mathematician Wiener who 
provided the following definition. 


Definition 10.1.5 (Ross, 1996) Brownian motion 

A stochastic process z (t) is a Brownian motion process if (i) z (0) = 0, (ii) the process has 
stationary independent increments and (iii) for every t > 0, z(t) is normally distributed 
with mean 0 and variance ot. 


The first condition z(0) = 0 is a normalization. Any z(t) that starts at, say 29, 
can be redefined as z(t) — z. The second condition says that for t4 > t3 > tz > tı the 
increment z (t4)—z (t3) , which is a random variable, is independent of previous increments, 
say z(t2) — z (tı). "Independent increments’ implies that Brownian motion is a Markov 
process. Assuming that we are in t3 today, the distribution of z (t4) depends only on 


10.1. Stochastic differential equations (SDEs) 227 


z (t3), ie. on the current state, and not on previous states like z (tı). Increments are 
said to be stationary if, according to the above definition of stationarity, the stochastic 
process X (t) = z(t) — z(t — s) where s is a constant, has the same distribution for any 
t. Finally, the third condition is the heart of the definition - z (t) is normally distributed. 
The variance increases linearly in time; the Wiener process is therefore non-stationary. 
Let us now define a stochastic process which plays also a major role in economics. 


Definition 10.1.6 Poisson process (adapted following Ross 1993, p. 210) 

A stochastic process q(t) is a Poisson process with arrival rate A if (i) q(0) = 0, (ii) the 
process has independent increments and (iii) the increment q(T) — q(t) in any interval 
of length T — t (the number of “jumps”) is Poisson distributed with mean [rT — t], i.e. 
q(T) — q(t) ~Poisson(A |r — t]). 


A Poisson process (and other related processes) are also sometimes called “counting 
processes” as q(t) counts how often a jump has occurred, i.e. how often something has 
happened. 

There is a close similarity in the first two points of this definition with the definition 
of Brownian motion. The third point here means more precisely that the probability that 
the process increases n times between t and T > t is given by 


ear Alr=d od (10.1.1) 


Pla(r) a(t) =n) = - 


We know this probability from the definition of the Poisson distribution in ch. 7.2.1. 
This is probably where the Poisson process got its name from. Hence, one could think 
of as many stochastic processes as there are distributions, defining each process by the 
distribution of its increments. 

The most common way to present Poisson processes is by looking at the distribution of 
the increment q (T) — q (t) over a very small time interval |t, 7T]. The increment q (T)—q (t) 
for T very close to t is usually expressed by dq (t). A stochastic process q (t) is then a 
Poisson process if its increment dq (t) is driven by 


0 with probability 1—Adt 
dq (t) = fo with Probability Adt > (10.1.2) 
where the parameter A is again called the arrival rate. A high A then means that the 
process jumps on average more often than with a low À. 

While this presentation is intuitive and widely used, one should note that the proba- 
bilities given in (10.1.2) are an approximation of the ones in (10.1.1) for 7—t = dt, i.e. for 
very small time intervals. We will return to this below in ch. 10.5.2, see Poisson process 
II. 

These stochastic processes (and other processes) can now be combined in various 
ways to construct more complex processes. These more complex processes can be well 
represented by stochastic differential equations (SDEs). 
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10.1.2 Stochastic differential equations 


The most frequently used SDEs include Brownian motion as the source of uncertainty. 
These SDEs are used to model for example the evolution of asset prices or budget con- 
straints of households. Other examples include SDEs with Poisson uncertainty used ex- 
plicitly in the natural volatility literature, in finance, labour markets, international macro 
or in other contexts mentioned above. Finally and more recently, Levy processes are used 
in finance as they allow for a much wider choice of properties of distributions of asset 
returns than, let us say, Brownian motion. We will now get to know examples for each 
type. 

For all Brownian motions that will follow, we will assume, unless explicitly stated 
otherwise, that increments have a standard normal distribution, i.e. Es |z (T) — z(t)] = 0 
and var; |z (T) — z (t)] =7—t. We will call this standard Brownian motion. It is therefore 
sufficient, consistent with most papers in the literature and many mathematical textbooks, 
to work with a normalization of ø in definition 10.1.5 of Brownian motion to 1. 


e Brownian motion with drift 


This is one of the simplest SDEs. It reads 
dx (t) = adt + bdz (t). (10.1.3) 


The constant a can be called drift rate, b? is sometimes referred to as the variance rate 
of x (t). In fact, ch. 10.5.4 shows that the expected increase of x(t) is determined by 
a only (and not by b). In contrast, the variance of x (7) for some future 7 > t is only 
determined by b. The drift rate a is multiplied by dt, a “short” time interval, the variance 
parameter b is multiplied by dz (t), the increment of the Brownian motion process z (t) 
over a small time interval. This SDE (and all the others following later) therefore consist 
of a deterministic part (the dt-term) and a stochastic part (the dz-term). 

An intuition for this differential equation can be most easily gained by undertaking a 
comparison with a deterministic differential equation. If we neglected the Wiener process 
for a moment (set b = 0), divide by dt and rename the variable as y, we obtain the simple 
ordinary differential equation 


y(t) =a (10.1.4) 


whose solution is y (t) = yo + at. When we draw this solution and also the above SDE for 
three different realizations of z (t), we obtain the following figure. 
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time 


Figure 10.1.1 The solution of the deterministic differential equation (10.1.4) and three 
realizations of the related stochastic differential equation (10.1.3) 


Hence, intuitively speaking, adding a stochastic component to the differential equation 
leads to fluctuations around the deterministic path. Clearly, how much the solution of 
the SDE differs from the deterministic one is random, i.e. unknown. Further below in 
ch. 10.5.4, we will understand that the solution of the deterministic differential equation 
(10.1.4) is identical to the evolution of the expected value of x (t) , i.e. y (t) = Eoz (t) for 
t>0. 


e Generalized Brownian motion (Ito processes) 


A more general way to describe stochastic processes is the following SDE 
dx (t) = a(a(t), z(t) ,t) dt +b (z (t), z(t), t)dz(t). (10.1.5) 


Here, one also refers to a(.) as the drift rate and to b?(.) as the instantaneous variance 
rate. Note that these functions can be stochastic themselves. In addition to arguments 
x(t) and time, Brownian motion z(t) can be included in these arguments. Thinking of 
(10.1.5) as a budget constraint of a household, an example could be that wage income or 
the interest rate depend on the current realization of the economy’s fundamental source 
of uncertainty, which is z (t). 


e Stochastic differential equations with Poisson processes 


Differential equations that are driven by a Poisson process can, of course, also be 
constructed. A very simple example is 


dx (t) = adt + bdq (t). (10.1.6) 


A realization of this path for æ (0) = zo is depicted in the following figure and can be 
understood very easily. As long as no jump occurs, i.e. as long as dq = 0, the variable 
x (t) follows dx (t) = adt which means linear growth, x (t) = zo +at. This is plotted as the 
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thin line. When q jumps, i.e. dq = 1, x(t) increases by b : writing dz (t) = #(t) — x(t), 
where Z(t) is the level of x immediately after the jump, and letting the jump be “very 
fast” such that dt = 0 during the jump, we have z(t) — x(t) =b- 1, where the 1 stems 
from dq (t) = 1. Hence, 

Z(t) =2(t) +b. (10.1.7) 


Clearly, the points in time when a jump occurs are random. A tilde (~) will always 
denote in what (and in various papers in the literature) follows the value of a quantity 
immediately after a jump. 


x(t) 
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Figure 10.1.2 An example of a Poisson process with drift (thick line) and a deterministic 
differential equation (thin line) 


In contrast to Brownian motion, a Poisson process contributes to the increase of the 
variable of interest: without the dq (t) term (i.e. for b = 0), x(t) would follow the thin 
line. With occasional jumps, x (t) grows faster. In the Brownian motion case of the figure 
before, realizations of x (t) remained “close to” the deterministic solution. This is simply 
due to the fact that the expected increment of Brownian motion is zero while the expected 
increment of a Poisson process is positive. 

Note that in the more formal literature, the tilde is not used but a difference is made 
between x (t) and x (t_) where t_ stands for the point in time an “instant” before t. (This 
is probably easy to understand on an intuitive level, thinking about it for too long might 
not be a good idea as time is continuous ...) The process x (t) is a so called cadlag process. 
The expression cadlag is an acronym from the french “continue a droite, limites à gauche”. 
That is, the paths of x (t) are continuous from the right with left limits. This is captured 
in the above figure by the black dots (continuous from the right) and the white circles 
(limits from the left). With this notation, one would express the change in x due to a 
jump by x(t) = x(t_) + b as the value of x to which b is added is the value of x before 
the jump. As the tilde-notation turned out to be relatively intuitive, we will follow it in 
what follows. 
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e A geometric Poisson process 


An further example would be the geometric Poisson process 
dz (t) =a (q (t), t) x (t) dt +b (q (t), t) x (t) dq (t). (10.1.8) 


Processes are usually called geometric when they describe the rate of change of some RV 
x(t), ie. dx (t) /x (t) is not a function of x (t). In this example, the deterministic part 
shows that x (t) grows at the rate of a (.) in a deterministic way and jumps by b (.) percent, 
when q (t) jumps. Note that in contrast to a Brownian motion SDE, a(.) here is not the 
average growth rate of x(t) (see below on expectations). 

Geometric Poisson processes as here are sometimes used to describe the evolution of 
asset prices in a simple way. There is some deterministic growth component a (.) and some 
stochastic component b (.). When the latter is positive, this could reflect new technologies 
in the economy. When the latter is negative, this equation could be used to model negative 
shocks like oil-price shocks or natural disasters. 


e Aggregate uncertainty and random jumps 


An interesting extension of a Poisson differential equation consists in making the ampli- 
tude of the jump random. Taking a simple differential equation with Poisson uncertainty 
as starting point, dA (t) = bA (t) dq (t) , where b is a constant, we can now assume that 
b(t) is governed by some distribution, i.e. 


dA (t) = b(t) A(t)dq(t), where b(t) ~ (4,07). (10.1.9) 


Assume that A (t) is total factor productivity in an economy. Then, A (t) does not change 
as long as dq (t) = 0. When q (t) jumps, A (t) changes by b(t), i.e. dA (t) = A(t)—A(t) = 
b(t) A(t), which we can rewrite as 


A(t) =(1+b(t)) A(t), Vt where q(t) jumps. 


This equation says that whenever a jump occurs, A (t) increases by b(t) percent, i.e. by 
the realization of the random variable b (t) . Obviously, the realization of b (t) matters only 
for points in time where q (t) jumps. 

Note that (10.1.9) is the stochastic differential equation representation of the evolution 
of the states of the economy in the Pissarides-type matching model of Shimer (2005), 
where aggregate uncertainty, here A (t) follows from a Poisson process. The presentation 
in Shimer’s paper is, “A shock hits the economy according to a Poisson process with arrival 
rate A, at which point a new pair (p’, s’) is drawn from a state dependent distribution.” 
(p. 34). Note also that using (10.1.9) and assuming large families such that there is no 
uncertainty from labour income left on the household level would allow to analyze the 
effects of saving and thereby capital accumulation over the business cycle in a closed- 
economy model with risk-averse households. The background for the saving decision 
would be ch. 11.1. 
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10.1.3 The integral representation of stochastic differential equa- 
tions 


Stochastic differential equations as presented here can also be represented by integral 
versions. This is identical to the integral representations for deterministic differential 
equations in ch. 4.3.3. The integral representation will be used frequently when computing 
moments of x (T). As an example, think of the expected value of x for some future point 
in time 7, when expectations are formed today in t, i.e. information until t is available, 
Ez (T). See ch. 10.5.4 or 11.1.6. 


e Brownian motion 


Consider a differential equation as (10.1.5). It can more rigorously be represented by 
its integral version, 


z (T)— z(t) = L a(x, s)ds + [ b(x, s)dz (s). (10.1.10) 


This version is obtain by first rewriting (10.1.5) as dz (s) = a (x, s) ds +b (x, s) dz (s) , i.e. 

by simply changing the time index from t to s (and dropping z (s) and writing x instead 

of z (s) to shorten notation). Applying then the integral f on both sides gives (10.1.10). 
This implies, inter alia, a “differentiation rule” 


d [fate s)ds + b(x, s)dz o| = d |x (T) — z (t)] = dz (T) 
= a(x, T)dT + b(x,T)dz (T). 
e Poisson processes 


Now consider a generalized version of the SDE in (10.1.6), with again replacing t by s, 
dx (s) =a (x (s) ,q (s))ds +b (zx (s) ,q(s)) dq (s). The integral representation reads, after 
applying f; to both sides, 


(r)—0(t)= f ala(s),a(s))as+ f o(e(s),4(s)) dale). 


This can be checked by computing the differential with respect to time T. 


10.2 Differentials of stochastic processes 


Possibly the most important aspect when working with stochastic processes in continu- 
ous time is that rules for computing differentials of functions of stochastic processes are 
different from standard rules. These rules are provided by various forms of Ito’s Lemma 
or change of variable formulas (CVF). Ito’s Lemma is a “rule” of how to compute dif- 
ferentials when the basic source of uncertainty is Brownian motion. The CVF provides 
corresponding rules when uncertainty stems from Poisson processes or Levy processes. 
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10.2.1 Why all this? 


“Computing differentials of functions of stochastic processes” sounds pretty abstract. Let 
us start with an example from deterministic continuous time setups which gives an idea 
about what the economic background for such differentials are. 

Imagine the capital stock of an economy follows K (t) = I (t)— ôK (t) , an ordinary dif- 
ferential equation (ODE) known from ch. 4. Assume further that total factor productivity 
grows at an exogenous rate of g, À (t) /A (t) = g. Let output be given by Y (A (t), K (t), L) 
and let us ask how output grows over time. The reply would be provided by looking at 
the derivative of Y (.) with respect to time, 


d dA(t) y aK O yy ab 


ae (A(t), K (t), L)=Ya dt YK di 


Alternatively, written as a differential, we would have 
dY (A(t), K (t), L) = Yad A (t) + YedK (t) + YzdL. 


We can now insert equations describing the evolution of TFP and capital, dA (t) and 
dK (t), and take into account that employment L is constant. This gives 


dY (A(t), K (t), L) = YagA (t) dt + Yx [I (t) — 6K (t)] dt + 0. 


Dividing by dt would give a differential equation that describes the growth of Y, i.e. Y (t). 
The objective of the subsequent sections is to provide rules on how to compute dif- 

ferentials, of which dY (A (t) , K (t) , L) is an example, in setups where K (t) or A (t) are 

described by stochastic DEs and not ordinary DEs as just used in this example. 


10.2.2 Computing differentials for Brownian motion 


We will now provide various versions of Ito’s Lemma. For formal treatments, see the 
references in “further reading” at the end of this chapter. 


e One stochastic process 


Lemma 10.2.1 Consider a function F (t,x) of the diffusion process x € R that is at 
least twice differentiable in x and once int. The diffusion process is described by dx (t) = 
a(x (t),2(t),t)dt+ 6 (z (t), z(t), t) dz(t) as in (10.1.5). The differential dF reads 


1 
dF = F;dt + F,dzr + 5 Fee (dex)’ (10.2.1) 


where (dx)? is computed by using 


dtdt = dtdz = dzdt = 0, dzdz = dt. (10.2.2) 
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The “rules” in (10.2.2) can intuitively be understood by thinking about the “length of a 
graph” of a function, more precisely speaking about the total variation. For differentiable 
functions, the variation is finite. For Brownian motion, which is continuous but not 
differentiable, the variation goes to infinity. As there is a finite variation for differentiable 
functions, the quadratic (co)variations involving dt in (10.2.2) are zero. The quadratic 
variation of Brownian motion, however, cannot be neglected and is given by dt. For details, 
see the further-reading chapter 10.6 on “mathematical background”. 

Let us look at an example. Assume that x (t) is described by a generalized Brownian 
motion as in (10.1.5). The square of dz is then given by 


(dx)? = a? (.) (dt)? + 2a(.) b(.) dtdz + b? (.) (dz)? = b? (.) dt, 


where the last equality uses the “rules” from (10.2.2). The differential of F (t,x) then 
reads 
1 
dF = F,dt + F,a (.) dt + F,b(.) dz + 5 rab” (.) dt 


= iF + Fa(.) + B o} dt + Fb (.) dz. (10.2.3) 


When we compare this differential with the “normal” one, we recognize familiar terms: 
The partial derivatives times deterministic changes, F; + Fra (.), would appear also in 
circumstances where «x follows a deterministic evolution. Put differently, for b(.) = 0 in 
(10.1.5), the differential dF reduces to { F; + Fa (.)} dt. Brownian motion therefore affects 
the differential dF in two ways: first, the stochastic term dz is added and second, maybe 
more “surprisingly”, the deterministic part of dF is also affected through the quadratic 
term containing the second derivative Fyz. 


e The lemma for many stochastic processes 
This was the simple case of one stochastic process. Now consider the case of many 
stochastic processes. Think of the price of many stocks traded on the stock market. We 


then have the following 


Lemma 10.2.2 Consider the following set of stochastic differential equations, 


dx, = adt + biidzı Fst bimdZm, 


d£n = andt + bnidzı +... + bnmdZm. 
In matrix notation, they can be written as 


dx = adt + bdz(t) 
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where 
Tı ay bii aa bim dz 
T= : 4 = : , b= : : , dz = 
Ta An, Oni «es On dZm 


Consider further a function F(t, x) from [0, oo| x R” to R with time t and the n processes 
in x as arguments. Then 


1 


where, as an extension to (10.2.2), 
dtdt = dtdz; = dz;dt = 0 and dzdz; = pijdt. (10.2.5) 


When all z; are mutually independent then p;; = 0 fori #7 and p,; = 1 for i= j. When 
two Brownian motions z; and z; are correlated, p;; is the correlation coefficient between 
their increments dz; and dz;. 


e An example with two stochastic processes 


Let us now consider an example for a function F (t, x,y) of two stochastic processes. 
As an example, assume that x is described by a generalized Brownian motion similar to 
(10.1.5), dx = a(t,x,y) dt + b(t,x,y) dz, and the stochastic process y is described by 
dy = c (t, x,y) dt + g (t,x, y) dz. Ito’s Lemma (10.2.4) gives the differential dF as 
1 
5 [Fre (dt)? + 2F,ydaedy + Fy, (dy)?| (10.2.6) 
Given the rule in (10.2.5), the squares and the product in (10.2.6) are 


(dx)? =b?(t,2,y)dt, (dy)? =9° (t,x, y) dt, ded = pyyb(.) 9 (.) dt, 


dF = F,dt + Fdz + F,dy 4 


where p, is the correlation coefficient of the two processes. More precisely, it is the 
correlation coefficient of the two normally distributed random variables that underlie the 
Wiener processes. The differential (10.2.6) therefore reads 


dF = Rat + a (.) Fedt + b (.) Frdza + c (.) Fydt + g (.) Fydzy 
1 
+ = |(Foub? (.) dt + 2PryFryb (-) g (.) dt + Fyyg? (.) dt) |] 


2 
z [r tal) Fs tel) Fyt 5 PO Fee + 2peyb (0) 9 (0) Fay + 9" () Fl at 
+ b(.) Frdz, + g (.) Fydzy (10.2.7) 


Note that this differential is almost simply the sum of the differentials of each sto- 
chastic process independently. The only term that is added is the term that contains 
the correlation coefficient. In other words, if the two stochastic processes were indepen- 
dent, the differential of a function of several stochastic processes equals the sum of the 
differential of each stochastic process individually. 
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e An example with one stochastic process and many Brownian motions 


A second example stipulates a stochastic process x(t) governed by dx = udt + 
“ui vidzi. This corresponds to n = 1 in the lemma above. When we compute the square 
of dx, we obtain 


(dx)? = (udt)? + 2u,dt [E2 vidz] + (Evida) =0 +04 (Evida), 
where the second equality uses (10.2.2). The differential of F (t,x) therefore reads from 
(10.2.4) 
1 
dF (t, x) = Fydt + Fe [udt + vide] + 5 Fee [Er ude]? 


1 
= {F,+ Fyu,} dt + 5 Fes [a vid]? + Fp Xe vidzi. 


Computing the De vdz] term requires to take potential correlations into account. For 
any two uncorrelated increments dz; and dz;, dz;dz; would from (10.2.5) be zero. When 
they are correlated, dz;dz; = p,;dt which includes the case of dzjdz; = dt. 


10.2.3 Computing differentials for Poisson processes 


When we consider the differential of a function of the variable that is driven by the Poisson 
process, we need to take the following CVFs into consideration. 


e One stochastic process 


Lemma 10.2.3 Let there be a stochastic process x(t) driven by Poisson uncertainty q (t) 
described by the following stochastic differential equation 


dz (t) = a(.) dt + b(.) dq (t). 
Consider the function F (t,x). The differential of this function is 
dF (t,x) = Fidt+ F,a(.)dt+{F (t,2+5(.)) —-F(t,2)} dq. (10.2.8) 


What was stressed before for Brownian motion is valid here as well: the functions a (.) 
and 6(.) in the deterministic and stochastic part of this SDE can have as arguments any 
combinations of q (t), x (t) and t or can be simple constants. 

The rule in (10.2.8) is very intuitive: the differential of a function is given by the 
“normal terms” and by a “jump term”. The “normal terms” include the partial derivatives 
with respect to time t and x times changes per unit of time (1 for the first argument and 
a(.) for x) times dt. Whenever the process q increases, x increases by the b(.). The 
“jump term” therefore captures that the function F (.) jumps from F (t,x) to F (t, z) = 
F (t,x +b(.)). 
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e Two stochastic processes 


Lemma 10.2.4 Let there be two independent Poisson processes qy and qy driving two 
stochastic processes x(t) and y (t), 


dx = a(.) dt + b(.) dq, dy = c(.)dt+q(.) dqy 
and consider the function F (x,y). The differential of this function is 


dF (x,y) = {Fra (.) + Fye(.)} dt + {F (£ + 6(.),.y) — F (a, y)} dae 
+ {F (x,y +g (-)) — F (x,y) } day. (10.2.9) 


Again, this “differentiation rule” consists of the “normal” terms and the “jump terms”. 
As the function F (.) depends on two arguments, the normal term contains two drift 
components, Fi,a(.) and F,c(.) and the jump term contains the effect of jumps in qz 
and in q}. Note that the dt term does not contain the time derivative F, (x,y) as in this 
example, F (x,y) is assumed not to be a function of time and therefore F, (x,y) = 0. In 
applications where F (.) is a function of time, the F;(.) would, of course, have to be taken 
into consideration. Basically, (10.2.9) is just the “sum” of two versions of (10.2.8). There 
is no additional term as the correlation term in the case of Brownian motion in (10.2.7). 
This is due to the fact that any two Poisson processes are, by construction, independent. 

Let us now consider a case that is frequently encountered in economic models when 
there is one economy-wide source of uncertainty, say new technologies arrive or commod- 
ity price shocks occur according to some Poisson process, and many variables in this 
economy (e.g. all relative prices) are affected simultaneously by this one shock. The CVF 
in situations of this type reads 


Lemma 10.2.5 Let there be two variables x and y following 
dx = a(.) dt + b(.) dq, dy = c(.) dt + g(.) dq, 


where uncertainty stems from the same q for both variables. Consider the function 
F(x,y). The differential of this function is 


dF (x,y) = {Fra(.) + Fye(.)} dt + {F (e+ b(.),y+9()) — F(a, y)} da. 


One nice feature about differentiation rules for Poisson processes is their very intuitive 
structure. When there are two independent Poisson processes as in (10.2.9), the change in 
F is given by either F (x + b(.),y) — F (x,y) or by F (x,y +g (.)) — F (x,y) , depending 
on whether one or the other Poisson process jumps. When both arguments x and y are 
affected by the same Poisson process, the change in F is given by F (x + b(.),y+g(.))— 
F (x,y), i.e. the level of F after a simultaneous change of both x and y minus the pre-jump 
level F (x,y). 
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e Many stochastic processes 


We now present the most general case. Let there be n stochastic processes x; (t) and 
define the vector x (t) = (x1 (t),...,2n (t))’ . Let stochastic processes be described by n 
SDEs 


where (3;; (.) stands for 6;; (t, x (t)). Each stochastic process zx; (t) is driven by the same 
m Poisson processes. The impact of Poisson process qj on x; (t) is captured by (3;; (.). 
Note the similarity to the setup for the Brownian motion case in (10.2.4). 


Proposition 10.2.1 Let there be n stochastic processes described by (10.2.10). For a 
once continuously differentiable function F (t,x), the process F (t,x) obeys 
dF (t, x (t)) = {£0 (.) + Bi Fo, (.) ai (.) f dt 
+ Ejh {F (t, (t) + 8; (-)) — F (t,x @)} daj, (10.2.11) 

where F; and F,,, i = 1,...,n, denote the partial derivatives of f with respect to t and 
xi, respectively, and 3, stands for the n-dimensional vector function (Oi its ae 

The intuitive understanding is again simplified by focusing on “normal” continuous 
terms and on “jump terms”. The continuous terms are as before and simply describe the 
impact of the a; (.) in (10.2.10) on F (.). The jump terms show how F'(.) changes from 
F (t, a (t)) to F (t, æ (t) + 8; (.)) when Poisson process j jumps. The argument z (t)+£; (.) 
after the jump of q; is obtained by adding 3;; to component z; in x, ie. x(t) + 8; () = 
(x + bijs T2 + Baj; vey Tn + Brg) $ 


10.2.4 Brownian motion and a Poisson process 


There are much more general stochastic processes in the literature than just Brownian 
motion or Poisson processes. This section provides a CVF for a function of a variable 
which is driven by both Brownian motion and a Poisson process. More general processes 
than just additive combinations are so-called Levy processes, which will be analyzed in 
future editions of these notes. 


Lemma 10.2.6 Let there be a variable x which is described by 
dz =a(.)dt+b(.)dz + g(.) dq (10.2.12) 
and where uncertainty stems from Brownian motion z and a Poisson process q. Consider 
the function F (t,x). The differential of this function is 
dF (t,x) = [r + Fra (.) + Fab o} dt + Frb (.)dz + {F (t,x +g(.))— F(t, x)} dq. 
(10.2.13) 


Note that this lemma is just a “combination” of Ito’s Lemma (10.2.3) and the CVF 
for a Poisson process from (10.2.8). For an arrival rate of zero, i.e. for dq = 0 at all times, 
(10.2.13) is identical to (10.2.3). For b(.) = 0, (10.2.13) is identical to (10.2.8). 
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10.3 Applications 


10.3.1 Option pricing 


One of the most celebrated papers in economics is the paper by Black and Scholes (1973) 
in which they derived a pricing formula for options. This section presents the first steps 
towards obtaining this pricing equation. The subsequent chapter 10.4.1 will complete the 
analysis. This section presents a simplified version (by neglecting jumps in the asset price) 
of the derivation of Merton (1976). The basic question is: what is the price of an option 
on an asset if there is absence of arbitrage on capital markets? 


e The asset and option price 


The starting point is the price S of an asset which evolves according to a geometric 
process 


E = adt + odz. (10.3.1) 


Uncertainty is modelled by the increment dz of Brownian motion. We assume that the 
economic environment is such (inter alia short selling is possible, there are no transaction 
costs) that the price of the option is given by a function F (.) having as arguments only 
the price of the asset and time, F(t, S (t)). The differential of the price of the option is 
then given from (10.2.1) by 


1 
dF = Frdt + FsdS + 5Fss [dS]. (10.3.2) 
As by (10.2.2) the square of dS is given by (dS)? = o7Sdt, the differential reads 


1 dF 
dF = [r +aS Fs + TN dt + oS Fsdz <= F = apdt + o pdz (10.3.3) 


where the last step defined 


F; + aS Fs + t0? 9?F F 
sete’ sa 2 graa (10.3.4) 


QF 
e Absence of arbitrage 
Now comes the trick - the no-arbitrage consideration. Consider a portfolio that consists 


of N; units of the asset itself, No options and N3 units of some riskless assets, say wealth 
in a savings account. The price of such a portfolio is then given by 


P= NS + NF + NM, 
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where M is the price of one unit of the riskless asset. The proportional change of the price 
of this portfolio can be expressed as (holding the N;s constant, otherwise Ito’s Lemma 
would have to be used) 


dP NıSdS | N2FdF | N3MdM 
P PS: PF P M’ 
Defining shares of the portfolio held in these three assets by 04 = N1 S/P and 62 = Na F/P, 


inserting option and stock price evolutions from (10.3.1) and (10.3.2) and letting the 
riskless asset M pay a constant return of r, we obtain 


dP/P = 0,adt + @,;0dz + O2ardt + O20 pdz + (1 — 6, — 02) rdt 
= {9; læ —r] + 02 [ar —r]+r}dt+ {010 + 020 F} dz. (10.3.5) 


Now assume someone chooses weights such that the portfolio no longer bears any risk 
O10 + Ooo F = 0. (10.3.6) 


The return of such a portfolio with these weights must then of course be identical to the 
return of the riskless interest asset, i.e. identical to r, 


P = = 
ria = bila — r] + 62 lar = r] + rlo o= A ee 


—020 F _ 
riskless 


If the return of the riskless portfolio did not equal the return of the riskless interest rates, 
there would be arbitrage possibilities. This approach is therefore called no-arbitrage 
pricing. 


e The Black-Scholes formula 


Finally, inserting ar and op from (10.3.4) yields the celebrated differential equation 
that determines the evolution of the price of the option, 


a-Tr F,+aSFs + $0°S?F 55 — rF 


1 
& -0° S* Fog +7rSFy —rF +F,=0. (10.3.7) 
o oS Fs 2 


Clearly, this equation does not to say what the price F of the option actually is. It only 
says how it changes over time and in reaction to S. But as we will see in ch. 10.4.1, this 
equation can actually be solved explicitly for the price of the option. Note also that we 
did not make any assumption so far about what type of option we are talking about. 


10.3.2 Deriving a budget constraint 


Most maximization problems require a constraint. For a household, this is usually the 
budget constraint. It is shown here how the structure of the budget constraint depends 
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on the economic environment the household finds itself in and how the CVF needs to be 
applied here. 

Let wealth at time t be given by the number n (t) of stocks a household owns times 
their price v (t), a(t) = n (t) v (t). Let the price follow a process that is exogenous to the 
household (but potentially endogenous in general equilibrium), 


dv (t) = av (t) dt + Bu (t) dq (t), (10.3.8) 


where a and 8 are constants. For 3 we require 3 > —1 to avoid that the price can 
become zero or negative. Hence, the price grows with the continuous rate a and at 
discrete random times it jumps by 8 percent. The random times are modeled by the 
jump times of a Poisson process q (t) with arrival rate A, which is the “probability” that 
in the current period a price jump occurs. The expected (or average) growth rate is then 
given by a + XG (see ch. 10.5.4). 

Let the household earn dividend payments, 7 (t) per unit of asset it owns, and labour 
income, w (t). Assume furthermore that it spends p(t) c(t) on consumption, where c (t) 
denotes the consumption quantity and p (t) the price of one unit of the consumption good. 
When buying stocks is the only way of saving, the number of stocks held by the household 
changes in a deterministic way according to 


n (t) T (t) + w (t) - p (t) c (t) 
v (t) 

When savings n (t)r (t) + w(t) — p(t) c(t) are positive, the number of stocks held by 

the household increases by savings divided by the price of one stock. When savings are 

negative, the number of stocks decreases. 

The change in the household’s wealth, i.e. the household’s budget constraint, is then 
given by applying the CVF to a(t) = n (t) v(t). The appropriate CVF comes from (10.2.9) 
where only one of the two differential equations shows the increment of the Poisson process 
explicitly. With F (x,y) = xy, we obtain 


a= {ro we ALOR wo} a 


+ {n (t) [v (t) + Bu (t)] — n (t) v (t)} dq (t) 
= {r(t) a(t) + w (t) — p(t) c(t)} dt + Ba (t) dq (t), (10.3.9) 


where the interest-rate is defined as 


dt. 


dn (t) = 


_ mt) 
r(t)= TC) +a. 
This is a very intuitive budget constraint: As long as the asset price does not jump, i.e., 
dq (t) = 0, the household’s wealth increases by current savings, r (t) a (t)+w (t)—p (t)c(t), 
where the interest rate, r (t), consists of dividend payments in terms of the asset price plus 
the deterministic growth rate of the asset price. If a price jump occurs, i.e., dq (t) = 1, 
wealth jumps, as the price, by 8 percent, which is the stochastic part of the overall 
interest-rate. Altogether, the average interest rate amounts to r (t) + AG (see ch. 10.5.4). 
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10.4 Solving stochastic differential equations 


Just as there are theorems on uniqueness and existence of solutions for ordinary differential 
equations, there are theorems for SDEs on these issues. There are also solution methods 
for SDEs. Here, we will consider some examples for solutions of SDEs. 

Just as for ordinary deterministic differential equations in ch. 4.3.2, we will simply 
present solutions and not show how they can be derived. Solutions of stochastic differential 
equations d (x (t)) are, in analogy to the definition for ODE, again time paths zx (t) that 
satisfy the differential equation. Hence, by applying Ito’s Lemma or the CVF, one can 
verify whether the solutions presented here are indeed solutions. 


10.4.1 Some examples for Brownian motion 


This section first looks at SDEs with Brownian motion which are similar to the ones that 
were presented when introducing SDEs in ch. 10.1.2: We start with Brownian motion 
with drift as in (10.1.3) and then look at an example for generalized Brownian motion in 
(10.1.5). In both cases, we work with SDEs which have an economic interpretation and 
are not just SDEs. Finally, we complete the analysis of the Black-Scholes option pricing 
approach. 


e Brownian motion with drift 1 


As an example for Brownian motion with drift, consider a representation of a produc- 
tion technology which could be called a “differential-representation” for the technology. 
This type of presenting technologies was dominant in early contributions that used contin- 
uous time methods under uncertainty but is sometimes still used today. A simple example 
is 

dY (t) = AKdt+oKdz(t), (10.4.1) 
where Y (t) is output in t, A is a (constant) measure of total factor productivity, K is the 
(constant) capital stock, ø is some variance measure of output and z is Brownian motion. 
The change of output at each instant is then given by dY (t). See “further reading” on 
references to the literature. 

What does such a representation of output imply? To see this, look at (10.4.1) as 
Brownian motion with drift, i.e. consider A, K, and ø to be a constant. The solution to 
this differential equation starting in t = 0 with Yo and z (0) is 


Y (t) = Y + AKt + oK [z (t) — z(0)]. 


To simplify an economic interpretation set Yo = z (0) = 0. Output is then given by 
Y (t) = (At+oz(t)) K. This says that with a constant factor input K, output in t is 
determined by a deterministic and a stochastic part. The deterministic part At implies 
linear (i.e. not exponential as is usually assumed) growth, the stochastic part oz (t) implies 
deviations from the trend. As z (t) is Brownian motion, the sum of the deterministic and 
stochastic part can become negative. This is an undesirable property of this approach. 
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To see that Y (t) is in fact a solution of the above differential-representation, just apply 
Ito’s Lemma and recover (10.4.1). 


e Brownian motion with drift 2 


As a second example and in an attempt to better understand why output can become 
negative, consider a standard representation of a technology Y (t) = A(t) K and let TFP 
A follow Brownian motion with drift, 


dA (t) = gdt + odz (t), 


where g and o are constants. What does this alternative specification imply? 
Solving the SDE yields A (t) = Ap+gt+oz (t) (which can again be checked by applying 
Ito’s lemma). Output is therefore given by 


Y (t) = (Ao + gt + 0z (t)) K = AoK + gKt +0 Kz (t) 
and can again become negative. 
e Geometric Brownian motion 


Let us now assume that TFP follows geometric Brownian motion process, 
dA (t) /A (t) = gdt + odz (t), (10.4.2) 


where again g and ø are constants. Let output continue to be given by Y (t) = A(t) K. 
The solution for TFP, provided an initial condition A (0) = Ao, is given by 


A(t) Agel a tet, (10.4.3) 


At any point in time t, the TFP level depends on time t and the current level of the 
stochastic process z(t). This shows that TFP at each point t in time is random and 
thereby unknown from the perspective of t = 0. Hence, a SDE and its solution describe 
the deterministic evolution of a distribution over time. One could therefore plot a picture 
of A (t) which in principle would look like the evolution of the distribution in ch. 7.4.1. 
Interestingly, and this is due to the geometric specification in (10.4.2) and impor- 
tant for representing technologies in general, TFP can not become negative. While 
Brownian motion z(t) can take any value between minus and plus infinity, the term 
ela 30 )tte2l®) ig always positive. With an AK specification for output, output is always 


positive, Y (t) = Apes 27 tte) K . In fact, it can be shown that output and TFP are 
lognormally distributed. Hence, the specification of TFP with geometric Brownian mo- 
tion provides an alternative to the differential-representation in (10.4.1) which avoids the 
possibility of negative output. 

The level of TFP at some future point in time t is determined by a deterministic part, 
(g — ło?) t, and by a stochastic part, oz (t). Apparently, the stochastic nature of TFP 
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has an effect on the deterministic term. The structure (the factor 1/2 and the quadratic 
term o”) reminds of the role the stochastic disturbance plays in Ito’s lemma. There as 
well (see e.g. (10.2.3)), the stochastic disturbance affects the deterministic component of 
the differential. As we will see later, however, this does not affect expected growth. In 
fact, we will show further below in (10.5.2) that expected output grows at the rate g (and 
is thereby independent of the variance parameter g). 

One can verify that (10.4.3) is a solution to (10.4.2) by using Ito’s lemma. To do so, 
we need to bring (10.4.2) into a form which allows us to apply the formulas which we got 
to know in ch. 10.2.2. Define x (t) = (g — 40°) t + oz (t) and A(t) = F (z (t)) = Age”. 
As a consequence, the differential for x (t) is a nice SDE, dx (t) = (g — 407) dt + odz (t). 
As this SDE is of the form as in (10.1.5), we can use Ito’s lemma from (10.2.3) and find 


dA (t) = dF (z (t)) = fr, (x (t)) E — 5 + Te (x (t)) o} dt + F; (x (t)) odz. 


Inserting the first and second derivatives of F (x (t)) yields 
1 
dA (t) = f a0 [s — 5" + T dt + Ape™adz & 


1 1 
dA(t)/A(t) = 19 z 50 + sor} dt + odz = gdt + odz, 
where the “iff’ reinserted A(t) = Ape” and divided by A(t). As A(t) from (10.4.3) 
satisfies the original SDE (10.4.2), A(t) is a solution of (10.4.2). 


e Option pricing 


Let us come back to the Black-Scholes formula for option pricing. The SDE derived 
above in (10.3.7) describes the evolution of the price F (t, S (t)) , where t is time and S (t) 
the price of the underlying asset at t. We now look at a European call option, i.e. an 
option which gives the right to buy an asset at some fixed point in time T, the maturity 
date of the option. The fixed exercise or strike price of the option, i.e. the price at which 
the asset can be bought is denoted by P. 

Clearly, at any point in time t when the price of the asset is zero, the value of the 
option is zero as well. This is the first boundary condition for our partial differential 
equation (10.3.7). When the option can be exercised at T and the price of the asset is S, 
the value of the option is zero if the exercise price P exceeds the price of the asset and 
S — P if not. This is the second boundary condition. 


F (t,0)=0, F(T,S) = max{0,5: = P}, 


In the latter case where S — P > 0, the owner of the option would in fact buy the asset. 
Given these two boundary conditions, the partial differential equation has the solution 


F (t, S (t)) = S (t) ¢ (d1) — Pe"? 414 (d2) (10.4.4) 
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where 


1 P 

Sm ~ 2 du, 
o(y) mie u 
8+ (29) C9) 


d cs t 
i ovT =t 


The expression F (t, S (t)) gives the price of an option at a point in time t < T where the 
price of the asset is S (t) . It is a function of the cumulative standard normal distribution 
o (y). For any path of S(t), time up to maturity T affects the option price through dı, 
də and directly in the second term of the above difference. More interpretation is offered 
by many finance textbooks. 


də = di —ovT — t. 


10.4.2 A general solution for Brownian motions 


Consider the linear stochastic differential equation for x (t), 
dx (t) = {a (t) x (t) +b (t)} dt + U@, {ci (t) x (t) + gi (t)} dz" (t) (10.4.5) 


where a(t), b(t), c; (t) and g; (t) are functions of time and z’ (t) are Brownian motions. 
The correlation coefficients of its increment with the increments of z/ (t) are pij- Let there 
be a boundary condition x (0) = zo. The solution to (10.4.5) is 


a (t) = eO (t) (10.4.6) 
where 
t 1 t 
y(t) = / fa (u) — z! w} du + am f ci (u) dz; (u), (10.4.7a) 
y(t) = zo + / e 49) {b (s) — &(s)} ds + on, f e 48) g; (s) dz; (s), (10.4.7b) 
Y (s) = a 1 pigs (8) c (8), (10.4.7c) 
® (5) = UH Ue pCa (8) gi ($) - (10.4.7d) 


To obtain some intuition for (10.4.6), we can first consider the case of certainty. For 
ci (t) = gi (t) = 0, (10.4.5) is a linear ODE and the solution is 


eb) = efo alu)du [zo + fs e~ Jo (“44 (s) ds]. This corresponds to the results we know from 


ch. 4.3.2, see (4.3.7). For the general case, we now prove that (10.4.6) indeed satisfies 
(10.4.5). 
In order to use Ito’s Lemma, write the claim (10.4.6) as 


x(t) = PO p(t) = fy, y E) (10.4.8) 
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where 
Ho= eo sw o} dt +0 ,; (t) dz; (t), (10.4.9a) 
dy (t) = e™™® {b (t) — & (t)} dt + UM, e™® g; (t) dzi (t). (10.4.9b) 
are the differentials of (10.4.7a) and (10.4.7b). 


In order to compute the differential of x(t) in (10.4.8), we have to apply the multidi- 
mensional Ito-Formula (10.2.6) where time is not an argument of f(.). This gives 


dx(t) = Oy (t) dy(t) + dy (t) + = {ep (t) [dy]? + 2e% [dydz] + 0 * [dz]*}. 
(10.4.10) 
As [dy]? = [Ec (t) dz; (t))° by (10.2.5) - all terms multiplied by dt equal zero - we obtain 
ay) Se ose) a0) at. (10.4.11) 
Further, again by (10.2.5), 


dydz = (Uc; (t) dz; (£)) (BR eg: dalt) 
igs ME Is per (2) galt) at: (10.4.12) 


Hence, reinserting (10.4.6), (10.4.9), (10.4.11) and (10.4.12) in (10.4.10) gives 
Ferre, fa Oe y o} dt + 52c (t) dz: (t) 


(t) — (t)} dt + £219: (t) dz: (t) 
[x(t ) pa ee Pigs (t) c NG; aC ) dt] + 2ye je Pigs (t ) gi (t) dt] 


- {b 
1 
2 


Rearranging gives 


de(t) = {x(t)a (t) + b(t)} dt + Ui, {æ(t)ci (t) + g: (t)} dz: (t) 


— ED (t) + o} dt 


+f FeCDRDrpya Wo 0 + BEER eye (t) a (eb a 
= {o(f)a(t) +d ()} dt + DE {ele + oO} dzs (D), 


where the last equality sign used (10.4.7c) and (10.4.7d). This is the original SDE in 
(10.4.5) which shows that the claim (10.4.6) is indeed a solution of (10.4.5). 
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10.4.3 Differential equations with Poisson processes 


The presentation of solutions and their verification for Poisson processes follows a similar 
structure as for Brownian motion. We start here with a geometric Poisson process and 
compare properties of the solution with the TFP Brownian motion case. We then look 
at a more general process - the description of a budget constraint - which extends the 
geometric Poisson process. One should keep in mind, as stressed already in ch. 4.3.3, 
that solutions to differential equations are different from the integral representation of 
e.g. ch. 10.1.3. 


e A geometric Poisson process 


Imagine that TFP follows a deterministic trend and occasionally makes a discrete 
jump. This is captured by a geometric description as in (10.4.2), only that Brownian 
motion is replaced by a Poisson process, 


dA (t) /A(t) = gdt + odq(t). (10.4.13) 


Again, g and o are constant with o > —1. 
The solution to this SDE is given by 


A(t) = Apestla@)—4(0)] nate), (10.4.14) 


Uncertainty does not affect the deterministic part here, in contrast to the solution (10.4.3) 
for the Brownian motion case. As before, TFP follows a deterministic growth component 
and a stochastic component, |q (t) — ¢(0)|In(1+o). The latter makes future TFP un- 
certain from the perspective of today. 

The claim that A (t) is a solution can be proven by applying the appropriate CVF. 
This will be done for the next, more general, example. 


e A budget constraint 
As a second example, we look at a dynamic budget constraint, 
da (t) = {r (t) a(t) + w (t) — c (t)} dt + Ba (t) dq. (10.4.15) 


Defining ®(s) = w (s) — c (s), the backward solution of (10.4.15) with initial condition 
a (0) = ao reads 


a(t) = ee ® Ç + [ e 45) (s) is (10.4.16) 
where y (t) is 
y(t) = | r (u) du + lq (t) — q(0)| ln (1 + 8). (10.4.17) 


Note that the solution in (10.4.16) has the same structure as the solution to a deterministic 
version of the differential equation (10.4.15) (which we would obtain for 6 = 0). In 
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fact, the structure of (10.4.16) is identical to the structure of (4.3.7). Put differently, 
the stochastic component in (10.4.15) only affects the discount factor y(t). This is not 
surprising - in a way - as uncertainty is proportional to a(t) and the factor 6 can be seen 
as the stochastic component of the interest payments on a (t). 

We have just stated that (10.4.16) is a solution. This should therefore be verified. To 
this end, define 


t 
z(t) = ao +f e YIB(s)ds, (10.4.18) 
0 


and write the solution (10.4.16) as a(t) = e¥z(t) where from (10.4.17) and (10.4.18) and 
Leibniz’ rule (4.3.3), 


dy(t) = r(t)dt + n(1+ B)dq(t), — dz(t) = eO B(4)dt. (10.4.19) 


We have thereby defined a function a (t) = F (y (t) , z (t)) where the SDEs describing the 
evolution of y (t) and z(t) are given in (10.4.19). This allows us to use the CVF (10.2.9) 
which then says 
dF F,r (t)dt + F,e-¥6 (t) dt + {F (y +1n (1 + 8), 2) — F (y, z)} dq & 
da(t) eM 2(t)r(t)dt + eOe MOG (t)dt + {ee Oma 24) — XM z(t) } dq 
{r (t) a(t) + ® (t)} dt + Ba (t) dq. 


This is the original differential equation. Hence, a (t) in (10.4.16) is a solution for (10.4.15). 


e The intertemporal budget constraint 


In stochastic worlds, there is also a link between dynamic and intertemporal budget 
constraints, just as in deterministic setups as in ch. 4.4.2. We can now use the solution 
(10.4.16) to obtain an intertemporal budget constraint. We first present here a budget 
constraint for a finite planning horizon and then generalize the result. 

For the finite horizon case, we can rewrite (10.4.16) as 


t t 
| ee (s) ds +e a(t) = ao + l ew (s) ds. (10.4.20a) 
0 0 


This formulation suggests a standard economic interpretation. Total expenditure over the 
planning horizon from 0 to ¢ on the left-hand side must equal total wealth on the right- 
hand side. Total expenditure consists of the present value of the consumption expenditure 
path c(s) and the present value of assets a(t) the household wants to hold at the end of 
the planning period, i.e. at t. Total wealth is given by initial financial wealth ap and the 
present value of current and future wage income w (s). All discounting takes place at the 
realized stochastic interest rate y (s) - no expectations are formed. 

In order to obtain an infinite horizon intertemporal budget constraint, the solution 
(10.4.16) should be written more generally - after replacing t by 7 and 0 by t - as 


a(t) = Ma, + i eH) (w (s) — c(s)) ds (10.4.21) 
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where y(T) is 


T 


ult) = r (u)du +1n (1 + 6) la (T) — q(t). (10.4.22) 


t 


Multiplying (10.4.21) by e~¥™ and rearranging gives 
a(t) e 9 + Í e¥)(e(s) — w(s))ds = a(t). 
t 


Letting T go to infinity, assuming a no-Ponzi game condition lim,_,., a(t) e~Y” = 0 and 
rearranging again yields 


The structure here and in (10.4.20a) is close to the one in the deterministic world. 
The present value of consumption expenditure needs to equal current financial wealth 
a(t) plus “human wealth”, i.e. the present value of current and future labour income. 
Here as well, discounting takes place at the “risk-corrected” interest rate as captured by 
y (T) in (10.4.22). Note also that this intertemporal budget constraint requires equality 
in realizations, not in expected terms. 


e A switch process 


Here are two funny processes which have an interesting and simple solution. Consider 
the initial value problem, 


da (t) = —2zx (t)dq (t), x (0) = zo. 


The solution is z (t) = (—1)" xo, i.e. x (t) oscillates between —zo and zo. 
Now consider the transformation y = y + x. It evolves according to 


dy = —2(y—y)dq, Yo = J + zo. 


Its solution is y (t) = g + (—1)°® zo and y (t) oscillates now between y — xo and 9 + £o. 

This could be nicely used for models with wage uncertainty, where wages are sometimes 
high and sometimes low. An example would be a matching model where labour income 
is w (i.e. Y + zo) when employed and b (i.e. Y — zo) when unemployed. The difference 
between labour income levels is then 279. The drawback is that the probability of finding 
a job is identical to the probability of losing it. 


10.5 Expectation values 


10.5.1 The idea 


What is the idea behind expectations of stochastic processes? When thinking of a sto- 
chastic process X (t), either a simple one like Brownian motion or a Poisson process or 
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more complex ones described by SDEs, it is useful to keep in mind that one can think of 
the value X (7) the stochastic process takes at some future point 7 in time as a “normal” 
random variable. At each future point in time X (7) is characterized by some mean, some 
variance, by a range etc. This is illustrated in the following figure. 

50004 


4000 4 


30004 


X(t) 


2000 7 


10001 we 


' 0.3 
time T 


Figure 10.5.1 The distribution of X (T) att =.4>t=0 


The figure shows four realizations of the stochastic process X (t) . The starting point is 
today in t = 0 and the mean growth of this process is illustrated by the dotted line. When 
we think of possible realizations of X (7) for T = .4 from the perspective of t = 0, we can 
imagine a vertical line that cuts through possible paths at 7 = .4. With sufficiently many 
realizations of these paths, we would be able to make inferences about the distributional 
properties of X (.4). As the process used for this simulation is a geometric Brownian 
motion process as in (10.4.2), we would find that X (.4) is lognormally distributed as 
depicted above. Clearly, given that we have a precise mathematical description of our 
process, we do not need to estimate distributional properties for X (T), as suggested by 
this figure, but we can explicitly compute them. 

What this section therefore does is provide tools that allow to determine properties 
of the distribution of a stochastic process for future points in time. Put differently, un- 
derstanding a stochastic process means understanding the evolution of its distributional 
properties. We will first start by looking at relatively straightforward ways to compute 
means. Subsequently, we provide some martingale results which allow us to then under- 
stand a more general approach to computing means and also higher moments. 
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10.5.2 Simple results 


e Brownian motion I 


Consider a Brownian motion process Z (T), T > 0. Let Z (0) = 0. From the definition 
of Brownian motion in ch. 10.1.1, we know that Z (7) is normally distributed with mean 
0 and variance g?r. Let us assume we are in t today and Z (t) 4 0. What is the mean 
and variance of Z (T) for T >t? 

We construct a stochastic process Z (T) — Z (t) which at T = t is equal to zero. Now 
imagining that t is equal to zero, Z (T) — Z (t) is a Brownian motion process as defined 
in ch. 10.1.1. Therefore, Æ; |Z (T) — Z (t)] = 0 and var: [Z (T) — Z (t)] = o? [r —t]. This 
says that the increments of Brownian motion are normally distributed with mean zero 
and variance o° [r — t|. Hence, the reply to our question is 


E,Z(r)= Z(t), varZ(r) = 07? [r-t]. 


For our convention o = 1 which we follow here as stated at the beginning of ch. 10.1.2, 
the variance would equal var;Z (T) = 7 — t. 


e Brownian motion II 


Consider again the geometric Brownian process dA (t) /A (t) = gdt+odz (t) describing 
the growth of TFP in (10.4.2). Given the solution A (t) = Age ar jea from (10.4.3), 
we would now like to know what the expected TFP level A (t) in t from the perspective 
of t = 0 is. To this end, apply the expectations operator Eo and find 


Eg A (t) = Ap Epel9- 27 ita) = Ael”) Epe”, (10.5.1) 


where the second equality exploited the fact that e(9-3")* is non-random. As z (t) is 
a standardized Brownian motion, z(t) ~ N (0,t), az (t) is normally distributed with 
N (0,07t). As a consequence (compare ch. 7.3.3, especially eq. (7.3.4)), e7* is lognor- 
mally distributed with mean E27 t, Hence, 


iL $ 


EpA (t) = Agel? )te2 = Age. (10.5.2) 


The expected level of TFP growing according to a geometric Brownian motion process 
grows at the drift rate g of the stochastic process. The variance parameter o does not 
affect expected growth. 

Note that we can also determine the variance of A (t) by simply applying the variance 
operator to the solution from (10.4.3), 


varoA (t) = varo (Aoele=20)te7t0) = A2e2(9- 2%") tvary (e770) , 
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We now make similar arguments to before when we derived (10.5.2). As oz (t) ~ N (0,07), 


oz(t 


the term e?* is lognormally distributed with, see (7.3.4), variance ert Ce — 1). Hence, 


we find 
vary A (t) Z A2e2(9-2%")teat (e E 1) = A2e(2(s-20°) +27) Ca a 1) 
Apes Ce — 1) ; 


This clearly shows that for any g > 0, the variance of A (t) increases over time. 
e Poisson processes I 


The expected value and variance of a Poisson process q (T) with arrival rate À are 


(t)+A[r—-t], r>t (10.5.3) 


As always, we compute moments from the perspective of t and T > t lies in the future. The 
expected value Eq (T) directly follows from the definition of a Poisson process in ch. 10.1.1, 
see (10.1.1). As the number of jumps is Poisson distributed with mean A [r — t] and the 
variance of the Poisson distribution is identical to its mean, the variance also follows from 
the definition of a Poisson process. Note that there are simple generalizations of the 
Poisson process where the variance differs from the mean (see ch. 10.5.4). 


e Poisson processes IT 


Poisson processes are widely used to understand events like finding a job, getting 
fired, developing a new technology, occurrence of an earthquake etc. One can model these 
situations by a Poisson process q (t) with arrival rate À. In these applications, the following 
questions are often asked. 

What is the probability that q(t) will jump exactly once between t and r? Given 
that by def. 10.1.6, ei Gree is the probability that q jumped n times by 7, i.e. 
q(T) = q(t) +n, this probability is given by P (q (T) = q(t) +1) = er") [r — t] . Note 
that this is a function which is non-monotonic in time 7, as illustrated in fig. 10.5.2. 

How can the expression P (q (T) = q (t) + 1) = e-*"~4) [r — t] be reconciled with the 
usual description of a Poisson process, as e.g. in (10.1.2), where it says that the probability 
of a jump is given by \dt? When we look at a small time interval dt, we can neglect e*“ 
as it is “very close to one” and we get P (dq (t) = 1) = Adt. As over a very small instant 
dt a Poisson process can either jump or not jump (it can not jump more than once in a 
small instant dt), the probability of no jump is therefore P (dq (t) = 0) = 1 — Adt. 
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probability ProbPoisson.nb 


time 


10 20 30 40 


Figure 10.5.2 The probability of one jump by time T for a high arrival rate (peak at 
around 3) and a low arrival rate (peak at around 11) 


What is the length of time between two jumps or events? Clearly, we do not know 
as jumps are random. We can therefore only ask what the distribution of the length of 
time between two jumps is. To understand this, we start from (10.1.1) which tells us that 
the probability that there is no jump by 7 is given by P [q (T) = q(t)] = e-*'”—4. The 
probability that there is at least one jump by 7 is therefore P [q (T) > q(t)] = 1—eA"~4. 
This term 1 — e~*!"~4] is the cumulative density function of an exponential distribution 
for a random variable T — t (see ex. 2). Hence, the length 7 — t between two jumps is 
exponentially distributed with density \e~*!7—4, 


e Poisson process IIT 


Following the same structure as with Brownian motion, we now look at the geometric 
Poisson process describing TFP growth (10.4.13). The solution was given in (10.4.14) 
which reads, slightly generalized with the perspective of t and initial condition A;, A (T) = 
Aresta) -aHa What is the expected TFP level in 7? 

Applying the expectation operator gives 


EA (T) = Aret- a0) Fr eal) n(L +2) (10.5.4) 


where, as in (10.5.1), we split the exponential growth term into its deterministic and 
stochastic part. To proceed, we need the following 


Lemma 10.5.1 (Posch and Wälde, 2006) Assume that we are int and form expectations 
about future arrivals of the Poisson process. The expected value of c*), conditional on t 
where q (t) is known, is 


E0) = EO el -DAT T>t, c k= const. 
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Note that for integer k, these are the raw moments of c10. 

Proof. We can trivially rewrite ch) = cha@cka@)—-a], At time t, we know the 
realization of q(t) and therefore E,ck™™ = ck E cku- Computing this expectation 
requires the probability that a Poisson process jumps n times between t and 7. Formally, 


eV (Alr Ht)" mei lied) (ca. Cates AW 


FE, cha) Z aes” a ZS, a 
eA ck F. n 
— e(K-I)Ar—-t) yoo AES AED (EME a) 
ae n! 
—cEA(T— n 
— e(ck-1A(r—t) roo eon TGAG —t)) — gl —DAG—2) 
fa n! í 
-ÀT n —ck Tt n 
where ene is the probability of q (T) = n and XX 4f 2 case) = 1 denotes the 


summation of the probability function over the whole support of the Poisson distribution 
which was used in the last step. For a generalization of this lemma, see ex. 8. m 

To apply this lemma to our case Eye) ™C+), we set c = e and k = In(1+o). 
Hence, E,et7) nate) = eln(1t+o)q(t) e(t —1)A[r—t] — elt) n(1+e) .oX[r-t] — ealt) n(1t+e)+oX[r-#] 
and, inserted in (10.5.4), the expected TFP level is 


E,A (r) = A,e9 t-a) In(1+0) o4(t) In(1t+o)+oA[r—t] — Are CTN, 


This is an intuitive result: the growth rate of the expected TFP level is driven both by 
the deterministic growth component of the SDE (10.4.13) for TFP and by the stochastic 
part. The growth rate of expected TFP is higher, the higher the determinist part, the 
g, the more often the Poisson process jumps on average (a higher arrival rate A) and the 
higher the jump (the higher ø). 

This confirms formally what was already visible in and informally discussed after fig. 
10.1.2 of ch. 10.1.2: A Poisson process as a source of uncertainty in a SDE implies that 
average growth is not just determined by the deterministic part of the SDE (as is the case 
when Brownian motion constitutes the disturbance term) but also by the Poisson process 
itself. For a positive ø, average growth is higher, for a negative ø, it is lower. 


10.5.3 Martingales 


Martingale is an impressive word for a simple concept. Here is a simplified definition 
which is sufficient for our purposes. For more complete definitions (in a technical sense), 
see “further reading”. 


Definition 10.5.1 A stochastic process x(t) is a martingale if, being in t today, the 
expected value of x at some future point T in time equals the current value of x, 


BaF) = Pb) TÈt. (10.5.5) 
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As the expected value of x(t) is x(t), Ez (t) = x(t), this can easily be rewritten 
as E; |x (7) — x (t)] = 0. This is identical to saying that x(t) is a martingale if the ex- 
pected value of its increments between now t and somewhere at 7 in the future is zero. 
This definition and the definition of Brownian motion imply that Brownian motion is a 
martingale. 

In what follows, we will use the martingale properties of certain processes relatively 
frequently, for example when computing moments. Here are now some fundamental ex- 
amples for martingales. 


e Brownian motion 


First, look at Brownian motion, where we have a central result useful for many applica- 
tions where expectations and other moments are computed. It states that f? f (z (s), s) dz (s), 
where f (.) is some function and z (s) is Brownian motion, is a martingale (Corollary 3.2.6, 
Øksendal, 1998, p. 33). Hence, 


E / ‘fos. (10.5.6) 


e Poisson uncertainty 


A similar fundamental result for Poisson processes exists. We will use in what follows 
the martingale property of various expressions ras ad ee T These 
expressions are identical to or special cases of te f (q(s),8 — Sif s) ds, of 
which Garcia and Griego (1994, theorem 5.3) have shown te : is a ee pone 


Hence, : | Tuad Ei iu sds ay (10.5.7) 


As always, A is the (constant) arrival rate of q (s). 


10.5.4 A more general approach to computing moments 


When we want to understand moments of some stochastic process, we can proceed in 
two ways. Either, a SDE is expressed in its integral version, expectations operators are 
applied and the resulting deterministic differential equation is solved. Or, the SDE is 
solved directly and then expectation operators are applied. We already saw examples for 
the second approach in ch. 10.5.2. We will now follow the first way as this is generally 
the more flexible one. We first start with examples for Brownian motion processes and 
then look at cases with Poisson uncertainty. 
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e The drift and variance rates for Brownian motion 


We will now return to our first SDE in (10.1.3), dx (t) = adt + bdz (t), and want to 
understand why a is called the drift term and b? the variance term. To this end, we 
compute the mean and variance of x (T) for T > t. 

We start by expressing (10.1.3) in its integral representation as in ch. 10.1.3. This 
gives 


— z(t) = f ads +f bdz(s) =alr —#] + b[z(r) — 2z ()]. (10.5.8) 
t t 
The expected value E;x (T) is then simply 


Eix(T) = 2 (t) + afr — t| + bE [2(7) —2z()| =x (t)+afr-t], (10.5.9) 


where the second step used the fact that the expected increment of Brownian motion is 
zero. As the expected value E;2(7) is a deterministic variable, we can compute the usual 
time derivative and find how the expected value of x (7), being today in t, changes the 
further the point 7 lies in the future, dE,2(7)/dr = a. This makes clear why a is referred 
to as the drift rate of the random variable z (.) . 

Let us now analyse the variance of x (7) where we also start from (10.5.8). The variance 
can from (7.3.1) be written as varsz(T) = E,x?(r) — [E,x(r)|°. In contrast to the term in 
(7.3.1) we need to condition the variance on t: If we computed the variance of x (7) from 
the perspective of some earlier t, the variance would differ - as will become very clear from 
the expression we will see in a second. Applying the expression from (7.3.1) also shows 
that we can look at any x (T) as a “normal” random variable: Whether z (T) is described 
by a stochastic process or by some standard description of a random variable, an x (rT) for 
any fix future point 7 in time has some distribution with corresponding moments. This 
allows us to use standard rules for computing moments. Computing first Fx? (T) gives 
by inserting (10.5.8) 


Eiz’ (T) 
= oF; {[x(t) +alr —t]+6[z(7) -—z (t)||"} 
= Ey {[e(t)+al[r - t]? + 2[e(t) + a(r -tb fel) — z (€)] + bll) — e} 


= (x(t) +a[r—d]?+2[x(t) + a(r — t)] bE, [2(7) — 2] + Er {Bb (alr) — 2 ()"} 
= [x(t)+al[r—t]? + PVE, {[z(r) -z 0P}, (10.5.10) 


where the last equality used again that the expected increment of Brownian motion is 
zero. As [E,a(r)|" = [x (t) + a|r — tl]? from (10.5.9), inserting (10.5.10) into the variance 
expression gives var,x(r) = bE; {[z(r) — z (t)|"} 

Computing the mean of the second moment gives 


E, {[z( Dl} = Ey {2° (T) — 22(r)z (t) + 2° (t)} = E: {27 (7)} — 2? (t) = variz (T), 
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where we used that z(t) is known in t and therefore non-stochastic, that E;z(T) = z(t) 
and equation (7.3.1) again. We therefore found that var,v(T) = b’var,z (T), the variance 
of x (T) is b? times the variance of z (T) . As the latter variance is given by var,z (T) = T—t, 
given that we focus on Brownian motion with standard normally distributed increments, 
we found 


var,x(T) = b? [r — t]. 


This equation shows why b is the variance parameter for x(.) and why b? is called its 
variance rate. The expression also makes clear why it is so important to state the current 
point in time, t in our case. If we were further in the past or 7 is further in the future, 
the variance would be larger. 


e Expected returns - Brownian motion 


Imagine an individual owns wealth a that is allocated to N assets such that shares 
in wealth are given by 6; = a;/a. The price of each asset follows a certain pricing rule, 
say geometric Brownian motion, and let’s assume that total wealth of the household, 
neglecting labour income and consumption expenditure, evolves according to 


da (t) = a(t) [rdt + DN bioidzi (t)] , (10.5.11) 


where r = ©_,0;r;. This is in fact the budget constraint (with w — c = 0) which will be 
derived and used in ch. 11.4 on capital asset pricing. Note that Brownian motions z; are 
correlated, i.e. dz;dz; = p,;dt as in (10.2.5). What is the expected return and the variance 
of holding such a portfolio, taking 0; and interest rates and variance parameters as given? 

Using the same approach as in the previous example we find that the expected return 
is simply given by r. This is due to the fact that the mean of the BMs z; (t) are zero. The 
variance of a(t) is left as an exercise. 


e Expected returns - Poisson 


Let us now compute the expected return of wealth when the evolution of wealth is 
described by the budget constraint in (10.3.9), da (t) = {r (t) a(t) + w (t) — p(t) c (t)} dt+ 
Ba (t) dq (t). When we want to do so, we first need to be precise about what we mean 
by the expected return. We define it as the growth rate of the mean of wealth when 
consumption expenditure and labour income are identical, i.e. when total wealth changes 
only due to capital income. 

Using this definition, we first compute the expected wealth level at some future point 
in time 7. Expressing this equation in its integral representation as in ch. 10.1.3 gives 


aa) -at= f Eal) +w(s)~p(s)e(s)}ds+ f Ba(s) da(s). 
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Applying the expectations operator yields 


E,a(r) - a(t) B | {r(s)a(s) +w(s)~p(s)e(s)}ds +B: f Baldl) 


= f Er (s) Ea (s) ds + BA n Ea (s) ds. (10.5.12) 


The second equality used an element of the definition of the expected return, i.e. w (s) = 
p(s)c(s), that r (s) is independent of a (s) and the martingale result of (10.5.7). When 
we define the mean of a (s) from the perspective of t by (s) = Era (s), this equation 
reads 


w(r)—a(t) =f Erud +AA f ul) ds 


Computing the derivative with respect to time T gives 


u (T) = Fur (T) w (7) + AB (T) & as = Esr (T) + BA. (10.5.13) 
Hence, the expected return is given by Er (T) + BX. 

The assumed independence between the interest rate and wealth in (10.5.12) is useful 
here but might not hold in a general equilibrium setup if wealth is a share of and the 
interest rate a function of the aggregate capital stock. Care should therefore be taken 
when using this result more generally. 


e Expected growth rate 


Finally, we ask what the expected growth rate and the variance of the price v (T) is 
when it follows the geometric Poisson process known from (10.3.8), du (T) = av (rT) dr + 


Bu (T) dq (T). 


The expected growth rate can be defined as EOS © Given that v (t) is known in 
t, we can write this expected growth rate as ea w where expectations are formed 


only with respect to the future price v (T). Given this expression, we can follow the usual 
approach. The integral version of the SDE, applying the expectations operator, is 


Ew (r) — v(t) =ak, | v(s)ds+ 8B f v(s)aa(s). 


Pulling the expectations operator inside the integral, using the martingale result from 
(10.5.7) and defining u (s) = Ev (s) gives 


ir) v(t) =a f nids pà f ul)ds 


The time-r derivative gives ù (T) = au (T) + Bàu (T) which shows that the growth rate 
of the mean of the price is given by œ + 6A. This is, as just discussed, also the expected 
growth rate of the price v (rT). 
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e Disentangling the mean and variance of a Poisson process 


Consider the SDE dv (t) /v (t) = adt + «Gdq (t) where the arrival rate of q (t) is given 
by \/k. The mean of u(r) is Ew (T) = v(t) expt?) and thereby independent of 
k. The variance, however, is given by varv (T) = [Ew (r)? apeg —1|. A mean 
preserving spread can thus be achieved by an increase of « : This increases the randomly 
occurring jump «6 and reduces the arrival rate \/« - the mean remains unchanged, the 
variance increases. 


e Computing the mean step-by-step for a Poisson example 


Let us now consider some stochastic processes X (t) described by a differential equation 
and ask what we know about expected values of X (T), where 7 lies in the future, i.e. 
T > t. We take as the first example a stochastic process similar to (10.1.8). We take as 
an example the specification for total factor productivity A, 


120) = odt + Bdr (t) — Bado: (t), (10.5.14) 


where a and (3, are positive constants and the arrival rates of the processes are given by 
some constant A; > 0. This equation says that TFP increases in a deterministic way at 
the constant rate a (note that the left hand side of this differential equation gives the 
growth rate of A(t)) and jumps at random points in time. Jumps can be positive when 
dq, = 1 and TFP increases by the factor 64, i.e. it increases by 3,%, or negative when 
dqz = 1, i.e. TFP decreases by 35%. 

The integral version of (10.5.14) reads (see ch. 10.1.3) 


A-A) = foals jds+ f BA s) dq (s )- f BAC s) dg (s) 


t 


=a f A(s)as+8, f A(s)das(s) - Ba | Ale) de(s); (10.5.15) 


t 


When we form expectations, we obtain 


T 


EA (T) — A(t) = ak; [ A(s)ds+ 6,E; [4 (s) dq, (s) — BaB, | A (s) dq (s) 


= aE; T A (s)ds + Bi; [A (s) ds — Boks | A(s) ds. 
i ' (10.5.16) 


where the second equality used the martingale result from ch. 10.5.3, i.e. the expression 
n (10.5.7). Pulling the expectations operator into the integral gives 


T 


E,A(r)-A(t)=a [ E,A(s)ds + ar | E,A(s) ds — Byr2 f EA (s) ds. 
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When we finally define mı (T) = EA (T) , we obtain a deterministic differential equation 
that describes the evolution of the expected value of A (T) from the perspective of t. We 
first have the integral equation 


mı (7) Al) =a f m (s)ds +61) f m (8) ds = B32 | m: (s) ds 


which we can then differentiate with respect to time T by applying the rule for differenti- 
ating integrals from (4.3.3), 


mı (T) = amı (T) + b1Amı (T) — BzA2mMı (T) 
= (a + ByA1 — b2A2) Mı (T). (10.5.17) 


We now immediately see that TFP does not increase in expected terms, more precisely 
EA (T) = A (t) ,ifa+61à1 = b2A2. Economically speaking, if the increase in TFP through 
the deterministic component a and the stochastic component 64 is “destroyed” on average 
by the second stochastic component 6, TFP does not increase. 


10.6 Further reading and exercises 
e Mathematical background 


There are many textbooks on differential equations, Ito calculus, change of variable 
formulas and related aspects in mathematics. One that is widely used is Øksendal (1998) 
and some of the above material is taken from there. A more technical approach is pre- 
sented in Protter (1995). Øksendal (1998, Theorem 4.1.2) covers proofs of some of the 
lemmas presented above. A much more general approach based on semi-martingales, and 
thereby covering all lemmas and CVFs presented here, is presented by Protter (1995). A 
classic mathematical reference is Gihman and Skorohod (1972). See also Goodman (2002) 
on an introduction to Brownian motion. 

A special focus with a detailed formal analysis of SDEs with Poisson processes can 
be found in Garcia and Griego (1994). They also provide solutions of SDEs and the 
background for computing moments of stochastic processes. Further solutions, applied 
to option pricing, are provided by Das and Foresi (1996). The CVF for the combined 
Poisson-diffusion setup in lemma, 10.2.6 is a special case of the expression in Sennewald 
(2007) which in turn is based on Øksendal and Sulem (2005). Øksendal and Sulem present 
CVFs for more general Levy processes of which the SDE (10.2.12) is a very simple special 
case. 

The claim for the solution in (10.4.6) is an “educated guess”. It builds on Arnold 
(1973, ch. 8.4) who provides a solution for independent Brownian motions. 
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e A less technical background 


A very readable introduction to stochastic processes is by Ross (1993, 1996). He writes 
in the introduction to the first edition of his 1996 book that his text is “a nonmeasure 
theoretic introduction to stochastic processes”. This makes this book highly accessible 
for economists. 

An introduction with many examples from economics is Dixit and Pindyck (1994). 
See also Turnovsky (1997, 2000) for many applications. Brownian motion is treated 
extensively in Chang (2004). 

The CVF for Poisson processes is most easily accessible in Sennewald (2007) or Sen- 
newald and Wilde (2006). Sennewald (2007) provides the mathematical proofs, Sen- 
newald and Wilde (2006) has a focus on applications. Proposition 10.2.1 is taken from 
Sennewald (2007) and Sennewald and Wälde (2006). 

The technical background for the t_ notation is the fact that the process x (t) is a so 
called cadlag process (“continue à droite, limites à gauche”), i.e. the paths of x(t) are 
continuous from the right with left limits. The left limit is denoted by x (t_) = limsy 2 (s). 
See Sennewald (2007) or Sennewald and Wilde (2006) for further details and references 
to the mathematical literature. 


e How to present technologies in continuous time 


There is a tradition in economics starting with Eaton (1981) where output is repre- 
sented by a stochastic differential equation as presented in ch. 10.4.1. This and similar 
representations of technologies are used by Epaulart and Pommeret (2003), Pommeret 
and Smith (2005), Turnovsky and Smith (2006), Turnovsky (2000), Chatterjee, Giuliano 
and Turnovsky (2004), and many others. It is well known (see e.g. footnote 4 in Grinols 
and Turnovsky (1998)) that this implies the possibility of negative Y. An alternative 
where standard Cobb-Douglas technologies are used and TFP is described by a SDE to 
this approach is presented in Walde (2005) or Wilde (2011). 


e Application of Poisson processes in economics 


The Poisson process is widely used in finance (early references are Merton, 1969, 1971) 
and labour economics (in matching and search models, see e.g. Pissarides, 2000, Burdett 
and Mortensen, 1998 or Moscarini, 2005). See also the literature on the real options ap- 
proach to investment (McDonald and Siegel, 1986, Dixit and Pindyck, 1994, Chen and 
Funke 2005 or Guo et al., 2005). It is also used in growth models (e.g. quality ladder 
models a la Aghion and Howitt, 1992 or Grossman and Helpman, 1991), in analyses 
of business cycles in the natural volatility tradition (e.g. Wilde, 2005), contract theory 
(e.g. Guriev and Kvasov, 2005), in the search approach to monetary economics (e.g. Kiy- 
otaki and Wright, 1993 and subsequent work) and many other areas. Further examples 
include Toche (2005), Steger (2005), Laitner and Stolyarov (2004), Farzin et al. (1998), 
Hassett and Metcalf (1999), Thompson and Waldo (1994), Palokangas (2003, 2005) and 
Venegas-Martinez (2001). 
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One Poisson shock affecting many variables (as stressed in lemma 10.2.5) was used 
by Aghion and Howitt (1992) in their famous growth model. When deriving the budget 
constraint in Appendix 1 of Walde (1999a), it is taken into consideration that a jump 
in the technology level affects both the capital stock directly as well as its price. Other 
examples include the natural volatility papers by Walde (2002, 2005) and Posch and 
Wälde (2006). 

“Disentangling the mean and variance of a Poisson process” is taken from Sennewald 
and Wilde (2006). An alternative is provided by Steger (2005) who uses two symmetric 
Poisson processes instead of one here. He obtains higher risk at an invariant mean by 
increasing the symmetric jump size. 


e Other 


Solutions to partial differential equations as in the option pricing example (10.4.4) 
are more frequently used e.g. in physics (see Black and Scholes, 1973, p. 644). Partial 
differential equations also appear in labour economics, however. See the Fokker-Planck 
equations in Bayer and Wilde (2010a,b). 

Definitions and applications of martingales are provided more stringently in e.g. Øk- 
sendal (1998) or Ross (1996). 
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Exercises Chapter 10 
Applied Intertemporal Optimization 


Stochastic differential equations and rules 
for differentiating 


1. Expectations 


(a) Assume the price of bread follows a geometric Brownian motion. What is the 
probability that the price will be 20% more expensive in the next year? 


(b) Consider a Poisson process with arrival rate A. What is the probability that 
a jump occurs only after 3 weeks? What is the probability that 5 jumps will 
have occurred over the next 2 days? 


2. Differentials of functions of stochastic processes I 
Assume x (t) and y (t) are two correlated Wiener processes. Compute d [æ (t) y (#)], 
d|a (t) /y (t)] and dln z (t). 


3. Differentials of functions of stochastic processes II 


(a) Show that with F = F (x,y) = ry and dx = f* (2, y) dt + g” (x,y) dq” and 
dy = f” (x,y) dt + g” (x, y) dq’, where q? and q” are two independent Poisson 
processes, dF = zdy + ydz. 


(b) Does this also hold for two Wiener processes q” and q”? 


4. Correlated jump processes 
Let q; (t) for i = 1,2,3 be three independent Poisson processes. Define two jump 
processes by qs (t) = qı (t) + q2 (t) and q, (t) = qı (t) + q3 (t). Given that qı (t) 
appears in both definitions, q, (t) and q, (t) are correlated jump processes. Let 
labour productivity in sector X and Y be driven by 


dA (t) = adt + Bdqz (t), 
dB (t) = ydt + ddq, (t) . 
Let GDP in a small open economy (with internationally given constant prices py 


and py) be given by 
Y (t) = PsA (t) Ls + pyB (t) Ly. 
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(a) Given an economic interpretation to equations dA (t) and dB (t), based on 
qi (t), = 1,2,3. 


(b) How does GDP evolve over time in this economy if labour allocation L, and 
Ly is invariant? Express the differential dY (t) by using dq, and dqy if possible. 


5. Deriving a budget constraint 
Consider a household who owns some wealth a = nv + neve, where n; denotes the 
number of shares held by the household and v; is the value of the share. Assume 
that the value of the share evolves according to 


dv; = a;vidt + B,vidq. 


Assume further that the number of shares held by the household changes according 


to 
Xi (Tiny + Tene + w — e) 


Ui 


dn; = 


dt, 
where y; is the share of savings used for buying stock 7. 


(a) Give an interpretation (in words) of the last equation. 


(b) Derive the household’s budget constraint. 


6. Option pricing 
Assume the price of an asset follows dS/S = adt + odz + dq (as in Merton, 1976), 
where z is Brownian motion and q is a Poisson process. This is a generalization of 
(10.3.1) where 6 = 0. How does the differential equation look like that determines 
the price of an option on this asset? 


7. Martingales 


(a) The weather tomorrow will be just the same as today. Is this a martingale? 


(b) Let z (s) be Brownian motion. Show that Y (t) defined by 


¥ sen |- / Odi) f PO a| gosa) 


is a martingale. 


(c) Show that X (t) defined by 


X() sex [f roro- [ Peas 


is also a martingale. 
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8. Expectations - Poisson 
Assume that you are in t and form expectations about future arrivals of the Poisson 
process q(t). Prove the following statement by using lemma 10.5.1: The number of 
expected arrivals in the time interval [r,s] equals the number of expected arrivals 
in a time interval of the length 7 — s for any s > t, 
E, (chla)—as)l) = E(u) GAG -s9) t> s> t, c, k= const. 


’ 


Hint: If you want to cheat, look at the appendix to Posch and Walde (2006). It is 
available for example on www. waelde.com/publications. 


9. Expected values 
Show that the growth rate of the mean of x (t) described by the geometric Brownian 
motion 


dx (t) = az (t) dt + ba (t) dz (t) 
is given by a. 
(a) Do so by using the integral version of this SDE and compute the increase of 
the expected value of x (t). 


(b) Do the same as in (a) but solve first the SDE and compute expectations by 
using this solution. 


(c) Compute the covariance of x (T) and z (s) for 7 > s >t. 


(d) Compute the density function f(xz(7)) for one specific point in time 7 > t. 
Hint: Compute the variance of x(7) and the expected value of x(7) as in (a) to 
(c). What type of distribution does x(7) come from? Compute the parameters 
u and o? of this distribution for one specific point in time 7 = 7. 


10. Expected returns 
Consider the budget constraint 


da (t) = {ra (t) + w — c(t)} dt + Ba (t) dz (t). 


(a) What is the expected return for wealth? Why does this expression differ from 
(10.5.13)? 


(b) What is the variance of wealth? 
11. “Differential-representation” of a technology 
(a) Consider the “differential-representation” of a technology, dY (t) = AKdt + 


oKdz(t), as presented in (10.4.1). Compute expected output of Y (7) for 
T >t and the variance of Y (rT). 
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(b) Consider a more standard representation by Y (t) = A (t) K and let TFP follow 
dA (t) /A (t) = gdt + odz (t) as in (10.4.2). What is the expected output level 
EY (T) and what is its variance? 


Solving stochastic differential equations 
Consider the differential equation 


dx (t) = [a (t) — x (t)] dt + cı (t) x (t) dz (t) + g2 (t) dza (t) 
where z,; and z2 are two correlated Brownian motions. 


(a) What is the solution of this differential equation? 


(b) Use Ito’s Lemma to show that your solution is a solution. 


Dynamic and intertemporal budget constraints - Brownian motion 
Consider the dynamic budget constraint 


da (T) = (r (vr) a (rT) + w (7) — p(T)c(T))dt + oa (T)dz (T), 
where z (T) is Brownian motion. 


(a) Show that the intertemporal version of this budget constraint, using a no-Ponzi 
game condition, can be written as 


J ep (7) e(r) dr = at f e Ow (1) dr (10.6.2) 
t t 


where the discount factor y (T) is given by 


e= (ro-3e) ds+ f ode(r). 


(b) Now assume we are willing to assume that intertemporal budget constraints 
need to hold in expectations only and not in realizations: we require only that 
agents balance at each instant expected consumption expenditure to current 
financial wealth plus expected labour income. Show that the intertemporal 
budget constraint then simplifies to 


z f e` FE (r(s)-0? as, (T)c(T)dT =a, + n f e7 JE (r()-0°)ds y (7) dr 
t t 


i.e. all stochastic terms drop out of the discount factor but the variance stays 
there. 


Chapter 11 


Infinite horizon models 


We now return to our main concern: How to solve maximization problems. We first look 
at optimal behaviour under Poisson uncertainty where we analyse cases for uncertain asset 
prices and labour income. We then switch to Brownian motion and look at capital asset 
pricing as an application. 


11.1 Intertemporal utility maximization under Pois- 
son uncertainty 


11.1.1 The setup 


Let us consider an individual that tries to maximize his objective function that is given 
by 

U (t) = z f elt—ty (e(r))dr. (11.1.1) 

t 

The structure of this objective function is identical to the one we know from deterministic 
continuous time models in e.g. (5.1.1) or (5.6.1): We are in t today, the time preference 
rate is p > 0, instantaneous utility is given by u (c (rT)). Given the uncertain environment 
the individual lives in, we now need to form expectations as consumption in future points 
T in time is unknown. 

When formulating the budget constraint of a household, we have now seen at various 
occasions that it is a good idea to derive it from the definition of wealth of a household. 
We did so in discrete time models in ch. 2.5.5 and in continuous time models in ch. 4.4.2. 
Deriving the budget constraint in stochastic continuous time models is especially impor- 
tant as a budget constraint in an economy where the fundamental source of uncertainty 
is Brownian motion looks very different from one where uncertainty stems from Poisson 
or Levy processes. For this first example, we use the budget constraint (10.3.9) derived 
in ch. 10.3.2, 

da (t) = {r (t) a(t) + w (t) — pe (t)} dt + Ba (t) dq (t), (11.1.2) 
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where the interest rate r(t) = œ + 7 (t) /v(t) was defined as the deterministic rate of 
change a of the price of the asset (compare the equation for the evolution of assets in 
(10.3.8)) plus dividend payments ~ (t) /v (t). We treat the price p here as a constant (see 
the exercises for an extension). Following the tilde notation from (10.1.7), we can express 
wealth a(t) after a jump by 

a(t)= (1+ B)a(t). (11.1.3) 


The budget constraint of this individual reflects standard economic ideas about budget 
constraints under uncertainty. As visible in the derivation in ch. 10.3.2, the uncertainty 
for this household stems from uncertainty about the evolution of the price (reflected in 
6B) of the asset he saves in. No statement was made about the evolution of the wage 
w (t). Hence, we take w(t) here as parametric, i.e. if there are stochastic changes, they 
all come as a surprise and are therefore not anticipated. The household does take into 
consideration, however, the uncertainty resulting from the evolution of the price v (t) 
of the asset. In addition to the deterministic growth rate a of u(t), v(t) changes in a 
stochastic way by jumping occasionally by 3 percent (again, see (10.3.8)). The returns to 
wealth a(t) are therefore uncertain and are composed of the “usual” r (t) a(t) term and 
the stochastic Ga (t) term. 
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11.1.2 Solving by dynamic programming 


We will solve this problem as before by dynamic programming methods. Note, however, 
that it is not obvious whether the above problem can be solved by dynamic program- 
ming methods. In principle, a proof is required that dynamic programming indeed yields 
necessary and sufficient conditions for an optimum. While proofs exist for bounded in- 
stantaneous utility function u (t) , such a proof did not exist until recently for unbounded 
utility functions. Sennewald (2007) extends the standard proofs and shows that dynamic 
programming can also be used for unbounded utility functions. We can therefore follow 
the usual three-step approach to dynamic programming here as well. 


e DP1: Bellman equation and first-order conditions 


The first tool we need to derive rules for optimal behavior is the Bellman equation. 
Defining the optimal program as V (a) = maxy,,)} U (t) subject to the constraint (11.1.2), 
this equation is given by (see Sennewald and Wilde, 2006, or Sennewald, 2007) 


pV (a(t)) = max fu (c(t)) + Z EdV (a o} . (11.1.4) 
The Bellman equation has this basic form for “most” maximization problems in continuous 
time. It can therefore be taken as the starting point for other maximization problems as 
well, independently, for example, of whether uncertainty is driven by Poisson processes, 
Brownian motion or Levy processes. We will see examples of related problems later (see 
ch. 11.1.4, ch. 11.2.2 or ch. 11.3.2) and discuss then how to adjust certain features of 
this “general” Bellman equation. In this equation, the variable a (t) represents the state 
variable, in our case wealth of the individual. See ch. 6.1 on dynamic programming in a 
deterministic continuous time setup for a detailed intuitive discussion of the structure of 
the Bellman equation. The discussion on “what is a state variable” of ch. 3.4.2 applies 
here as well. 

Given the general form of the Bellman equation in (11.1.4), we need to compute the 
differential dV (a (t)) . Given the evolution of a (t) in (11.1.2) and the CVF from (10.2.8), 
we find 

dV (a) = V' (a) {ra + w — pc} dt + {V (a + Ba) — V (a)} dq. 


In contrast to the CVF notation in for example (10.2.8), we use here and in what follows 
simple derivative signs like V’ (a) as often as possible in contrast to for example V, (a). 
This is possible as long as the functions, like the value function V (a) here, have one 
argument only. Forming expectations about dV (a (t)) is easy and they are given by 


EdV (a(t)) = V' (a) {ra + w — pe} dt + {V (a) — V (a)} Bidg. 


The first term, the “dt-term” is known in t: The current state a (t) and all prices are known 
and the shadow price V” (a) is therefore also known. As a consequence, expectations need 
to be applied only to the “dq-term”. The first part of the “dq-term”, the expression 
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V ((14+ 8)a) — V (a) is also known in t as again a(t), parameters and the function V 
are all non-stochastic. We therefore only have to compute expectations about dq. From 
(10.5.3), we know that E; |q (T) — q (t)] = A[7 — t]. Now replace q(T) — q(t) by dq and 
T —t by dt and find E,dq = Adt. The Bellman equation therefore reads 


pV (a) = {u(c(t)) +V (a) [ra + w — pe] +A[V (14+ B)a)—V(a)]}. (11.1.5) 


Note that forming expectations the way just used is, say, informal. Doing it in the more 
stringent way introduced in ch. 10.5.4 would, however, lead to identical results. 
The first-order condition is 
u' (c) = V' (a)p. (11.1.6) 


As always, (current) utility from an additional unit of consumption wu’ (c) must equal 
(future) utility from an additional unit of wealth V” (a), multiplied by the price p of the 
consumption good, i.e. by the number of units of wealth for which one can buy one unit 
of the consumption good. 


e DP2: Evolution of the costate variable 


In order to understand the evolution of the marginal value V’ (a) of the optimal pro- 
gram, i.e. the evolution of the costate variable, we need to (i) compute the partial deriv- 
ative of the maximized Bellman equation with respect to assets and (ii) compute the 
differential dV’ (a) by using the CVF and insert the partial derivative into this expres- 
sion. These two steps correspond to the two steps in DP2 in the deterministic continuous 
time setup of ch. 6.1. 

(i) In the first step, we state the maximized Bellman equation from (11.1.5) as the 
Bellman equation where controls are replaced by their optimal values, 


pV (a) = u (c (a)) + V" (a) [ra + w — pe (a)] + A [V (@) — V (a). 


We then compute again the derivative with respect to the state - as in discrete time and 
deterministic continuous time setups - as this gives us an expression for the shadow price 
V’ (a). In contrast to the previous emphasis on Ito’s Lemmas and CVFs, we can use for 
this step standard rules from algebra as we compute the derivative for a given state a - the 
state variable is held constant and we want to understand the derivative of the function 
V (a) with respect to a. We do not compute the differential of V (a) and ask how the value 
function changes as a function of a change in a. Therefore, using the envelope theorem, 


pV' (a) = V' (a)r + V” (a) [ra +w — pc] + A[V’ (4) [1 + 8] — VV’ (a)]. (11.1.7) 


We used here the definition of @ given in (11.1.3). 
(ii) In the second step, the differential of the shadow price V’ (a) is computed. Here, 
we do need a change of variable formula. Hence, given the evolution of a (t) in (11.1.2), 


dV' (a) = V” (a) [ra + w — pc] dt + [V (à) — V"(a)] dq. (11.1.8) 
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Finally, replacing V” (a) [ra + w — pc] in (11.1.8) by the same expression from (11.1.7) 
gives 


dv" (a) = {(p— r) V’ (a) — à [V" (@) [L + 8] — V" (a)]} dt + {V" (@) — V” (a)} dq. 
e DP3: Inserting first-order conditions 


Finally, we can replace the marginal value by marginal utility from the first-order 
condition (11.1.6). In this step, we employ that p is constant and therefore dV’ (a) = 
p ‘du’ (c). Hence, the rule describing the evolution of marginal utility reads 


du! (c) = {(o —r) u’ (c) — Alu’ (©) [1 + 8] — w (c)]} dt + {u' (ë) — ul (c)} da. (11.1.9) 


Note that the constant price p dropped out. This rule shows how marginal utility changes 
in a deterministic and stochastic way. 


11.1.3 The Keynes-Ramsey rule 


The dynamic programming approach provided us with an expression in (11.1.9) which 
describes the evolution of marginal utility from consumption. While there is a one-to-one 
mapping from marginal utility to consumption which would allow some inferences about 
consumption from (11.1.9), it would nevertheless be more useful to have a Keynes-Ramsey 
rule for optimal consumption itself. 


e The evolution of consumption 


If we want to know more about the evolution of consumption, we can use the CVF 
formula as follows. Let f (.) be the inverse function for u’, ie. f (u’(c)) = c, and apply 
the CVF to f (u’ (c)). This gives 


df(u'(c)) = F (w (o) {(e—r)u'(c) — Afu (©) [1 + 8] — w (c)]} dt 
+{f u (2) — F (uw (c))} da. 


As f(u (c)) = c, we know that f (u' (č)) = č and f’ (w (č)) = Dii = HG = WO 
Hence, 
de = —{(p-r)ul() — A lu (E [L + 8] — u (oat + {ë — c} da € 


u! (c) 


AO) gg fr APA = ) [1+ £] - 1 \ poe {č — c} dq. (11.1.10) 


w (c) u’ (c) u’ (c) 


This is the Keynes-Ramsey rule that describes the evolution of consumption under optimal 
behaviour for a household that faces interest rate uncertainty resulting from Poisson 
processes. This equation is useful to understand, for example, economic fluctuations in a 
natural volatility setup. It corresponds to its deterministic pendant in (5.6.4) in ch. 5.6.1: 
By setting À = 0 here (implying dq = 0), noting that we treated the price as a constant 
and dividing by dt, we obtain (5.6.4). 
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e A specific utility function 


Let us now assume that the instantaneous utility function is given by the widely used 
constant relative risk aversion (CRRA) utility function, 


c(t) 7-1 


WCF) = mares a 0. (11.1.11) 


Then, the Keynes-Ramsey rule becomes 
dc CNI Č 
o% = fr EES [a - B) (5) 5 1 \ tof? = i} dq. (11.1.12) 


The left-hand side gives the proportional change of consumption times ø, the inverse of the 
intertemporal elasticity of substitution o~t. This corresponds to oé/c in the deterministic 
Keynes-Ramsey rule in e.g. (5.6.5). Growth of consumption depends on the right-hand 
side in a deterministic way on the usual difference between the interest rate and time 
preference rate plus the “A—term” which captures the impact of uncertainty. When we 
want to understand the meaning of this term, we need to find out whether consumption 
jumps up or down, following a jump of the Poisson process. When ( is positive, the 
household holds more wealth and consumption increases. Hence, the ratio c/é is smaller 
than unity and the sign of the bracket term (1 + 8) (<)° — 1 is qualitatively unclear. If it 
is positive, consumption growth is faster in a world where wealth occasionally jumps by 
B percent. 

The dq-term gives discrete changes in the case of a jump in q. It is, however, tautolog- 
ical: When q jumps and dq = 1 and dt = 0 for this small instant of the jump, (11.1.12) 
says that odc/c on the left-hand side is given by ø {é/c — 1} on the right hand side. As the 
left hand side is by definition of dc given by ø [č — c] /c, both sides are identical. Hence, 
the level of ¢ after the jump needs to be determined in an alternative way. 


11.1.4 Optimal consumption and portfolio choice 


This section analyses a more complex maximization problem than the one presented in 
ch. 11.1.1. In addition to the consumption-saving trade-off, it includes a portfolio choice 
problem. Interestingly, the solution is much simpler to work with as ¢ can explicitly be 
computed and closed-form solutions can easily be found. 


e The maximization problem 


Consider a household that is endowed with some initial wealth aọ > 0. At each instant, 
the household can invest its wealth a(t) in both a risky and a safe asset. The share of 
wealth the household holds in the risky asset is denoted by 0 (t). The price vı (t) of one 
unit of the risky asset obeys the SDE 


dv, (t) = rıvı (t) dt + Buy (t) dq (t), (11.1.13) 
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where rı € R and 8 > 0. That is, the price of the risky asset grows at each instant with a 
fixed rate rı and at random points in time it jumps by 8 percent. The randomness comes 
from the well-known Poisson process q (t) with arrival rate A. The price və (t) of one unit 
of the safe asset is assumed to follow 


dvs (t) = T2V2 (t) dt, (11.1.14) 


where rə > 0. Let the household receive a fixed wage income w and spend c(t) > 0 on 
consumption. Then, in analogy to ch. 10.3.2, the household’s budget constraint reads 


da (t) = {[0 (t)rı + (1—0 (t)) re] a(t) + w — c (t)} dt + 60 (t)a (t)dq (t). (11.1.15) 


We allow wealth to become negative but we could assume that debt is always covered 
by the household’s lifetime labour income discounted with the safe interest rate ro, i.e. 
a(t) > —w/ro. 

Let intertemporal preferences of households be identical to the previous maximization 
problem - see (11.1.1). The instantaneous utility function is again characterized by CRRA 
as in (11.1.11), u(c) = (ct — 1) / (1 — o) . The control variables of the household are the 
nonnegative consumption stream {c(t)} and the share {@ (t)} held in the risky asset. To 
avoid a trivial investment problem, we assume 


Tı < T2 < rı +A. (11.1.16) 


That is, the guaranteed return of the risky asset, r1, is lower than the return of the riskless 
asset, r2, whereas, on the other hand, the expected return of the risky asset, rı + A, shall 
be greater than r2. If rı was larger than r2, the risky asset would dominate the riskless 
one and no one would want to hold positive amounts of the riskless asset. If rə exceeded 
rı + AG, the riskless asset would dominate. 


e DP1: Bellman equation and first-order conditions 


Again, the first step of the solution of this maximization problem requires a Bellman 
equation. Define the value function V again as V (a (t)) = maxy.,),9(7)} U (t). The basic 
Bellman equation is taken from (11.1.4). When computing the differential dV (a (t)) and 
taking into account that there are now two control variables, the Bellman equation reads 


pV(a) = max {u(c) + [(@r1 + (1-4) re)at+w—clV' (a) +A[V (a) -V (a)}}, 


e(t),0(t) 

(11.1.17) 
where & = (1+ 08)a denotes the post-jump wealth if at wealth a a jump in the risky 
asset price occurs. The first-order conditions which any optimal path must satisfy are 
given by 

u' (c) = V' (a) (11.1.18) 


and 
V' (a) (rı — 72) a+ AV’ (à) Ba = 0. (11.1.19) 
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While the first first-order condition equates as always marginal utility with the shadow 
price, the second first-order condition determines optimal investment of wealth into assets 
1 and 2, i.e. the optimal share 0. The latter first-order condition contains a deterministic 
and a stochastic term and households hold their optimal share if these two components 
just add up to zero. Assume, consistent with (11.1.16), that rı < r2. If we were in a 
deterministic world, i.e. A = 0, households would then only hold asset 2 as its return is 
higher. In a stochastic world, the lower instantaneous return on asset 1 is compensated by 
the fact that, as (11.1.13) shows, the price of this asset jumps up occasionally by the per- 
centage 3. Lower instantaneous returns rı paid at each instant are therefore compensated 
for by large occasional positive jumps. 

As this first-order condition also shows, returns and jumps per se do not matter: The 
difference rı — rg in returns is multiplied by the shadow price V’ (a) of capital and the 
effect of the jump size times its frequency, \Z, is multiplied by the shadow price V” (à) 
of capital after a jump. What matters for the household decision is therefore the impact 
of holding wealth in one or the other asset on the overall value from behaving optimally, 
i.e. on the value function V (a). The channels through which asset returns affect the 
value function is first the impact on wealth and second the impact of wealth on the value 
function i.e. the shadow price of wealth. 

We can now immediately see why this more complex maximization problem yields 
simpler solutions: Replacing in equation (11.1.19) V’ (a) with wu’ (c) according to (11.1.18) 
yields for a # 0 

tt (Č)  r2=rı 

u’ (c) AB’ 
where č denotes the optimal consumption choice for a. Hence, the ratio for optimal 
consumption after and before a jump is constant. If we assume, for example, a CRRA 
utility function as in (11.1.11), this jump is given by 


7 1/o 
== ( Ag ) (11.1.21) 


T2 — 11 


(11.1.20) 


No such result on relative consumption before and after the jump is available for the 
maximization problem without a choice between a risky and a riskless asset. 

Since by assumption (11.1.16) the term on the right-hand side is greater than 1, 
this equation shows that consumption jumps upwards if a jump in the risky asset price 
occurs. This result is not surprising, as, if the risky asset price jumps upwards, so does 
the household’s wealth. 


e DP2: Evolution of the costate variable 


In the next step, we compute the evolution of V’ (a (t)), the shadow price of wealth. As- 
sume that V is twice continuously differentiable. Then, due to budget constraint (11.1.15), 
the CVF from (10.2.8) yields 

dV" (a) = {lr + (1-6) raja +w — c} V” (a) dt 
+ {V' (a) — V' (a)y dq (t). (11.1.22) 
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Differentiating the maximized Bellman equation yields under application of the envelope 
theorem 
pV’ (a) = {[6r1 + (1-4) relatw—c}V"(a) 
+ {0r + [1 — A] ro} V’ (a) + A {V (4) [1 + 06] — V"(a)}. 


Rearranging gives 
{[6r; + (1-6) re]a+w—c}V"(a) 
= {p — [Ori + (1 — 8) r2]} V'(a) — A{V" (4) [L + 06] — V’ (a)} . 
Inserting this into (11.1.22) yields 
(0) = | e ee gy pate a: 
e DP3: Inserting first-order conditions 


Replacing V” (a) by wu’ (c) following the first-order condition (11.1.18) for optimal con- 
sumption, we obtain 


rey J {eo — [Ori + 1 — 8) re]} u (c) wu (2)—u! (e 
al (=| OT tae @ tay pet O-w Ola, 


Now applying the CVF again to f (x) = (u’)~' (x) and using (11.1.20) leads to the Keynes- 
Ramsey rule for general utility functions u, 


_u'(c) Tati u” (c) 
u’ (c) AB a! (c) 


As č is also implicitly determined by (11.1.20), this Keynes-Ramsey rule describes the 
evolution of consumption under Poisson uncertainty without the č term. This is the crucial 
modelling advantage of introducing an additional asset into a standard consumption- 
saving problem. Apart from this simplification, the structure of this Keynes-Ramsey rule 
is identical to the one in (11.1.10) without a second asset. 


de={6n+[1-6]m—p+A] +65] -1] bar {é—chdq(t). 


e A specific utility function 


For the CRRA utility function as in (11.1.11), the elimination of č becomes even 
simpler and we obtain with (11.1.21) 


saath] jae (GR) "40. 


The optimal change in consumption can thus be expressed in terms of well-known para- 
meters. As long as the price of the risky asset does not jump, optimal consumption grows 
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constantly by the rate [rə — À (1 — am) — p| /o. The higher the risk-free interest rate, 
rg, and the lower the guaranteed interest rate of the risky asset, rı, the discrete growth 
rate, 3, the probability of a price jump, A, the time preference rate, p, and the risk aversion 
parameter, g, the higher becomes the consumption growth rate. If the risky asset price 
; ; . i 1/0 
jumps, consumption jumps as well to its new higher level c(t) = [(A8) / (r2 — rı) l e(t). 
Here the growth rate depends positively on à, 8, and r1, whereas r2 and o have a negative 
influence. 


11.1.5 Other ways to determine č 


The question of how to determine č without an additional asset is in principle identical 
to determining the initial level of consumption, given a deterministic Keynes-Ramsey 
rule as in for example (5.1.6). Whenever a jump in c following (11.1.12) occurs, the 
household faces the issue of how to choose the initial level of consumption after the jump. 
In principle, the level č is therefore pinned down by some transversality condition. In 
practice, the literature offers two ways, as to how č can be determined. 


e One asset and idiosyncratic risk 


When households determine optimal savings only, as in our setup where the only 
first-order condition is (11.1.6), č can be determined (in principle) if we assume that the 
value function is a function of wealth only - which would be the case in our household 
example if the interest rate and wages did not depend on q. This would naturally be the 
case in idiosyncratic risk models where aggregate variables do not depend on individual 
uncertainty resulting from q. The first-order condition (11.1.6) then reads wu’ (c) = V’ (a) 
(with p = 1). This is equivalent to saying that consumption is a function of the only state 
variable, i.e. wealth a, c = c (a). An example for c (a) is plotted in the following figure. 


ao ay ag a 


Figure 11.1.1 Consumption c as a function of wealth a 
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As consumption c does not depend on q directly, we must be at the same consumption 
level c(a), no matter “how we got there”, i.e. no matter how many jumps took place 
before. Hence, if we jump from some a, to dg because dq = 1, we are at the same 
consumption level c (a2) as if we had reached az smoothly without jump. The consumption 
level č after a jump is therefore the consumption level that belongs to the asset level after 
the jump according to the policy function c (a) plotted in the above figure, č = c (a2). See 
Schlegel (2004) for a numerical implementation of this approach. 


e Finding value functions 


A very long tradition exists in economics where value functions are found by an “edu- 
cated guess”. Experience tells us - based on first examples by Merton (1969, 1971) - what 
value functions generally look like. It is then possible to find, after some attempts, the 
value function for some specific problem. This then implies explicit - so called closed-form 
solutions - for consumption (or any other control variable). For the saving and investment 
problem in ch. 11.1.4, Sennewald and Wilde (2006, sect. 3.4) presented a value function 
and closed-form solutions for a Æ 0 of the form 


l-o 
Ja 2| -=p z w 
vies" k a 2 i e=vla+2], o-(( ae ) =i Tra 


ba 


Consumption is a constant share (4 is a collection of parameters) out of the total wealth, 
i.e. financial wealth a plus “human wealth” w/r (the present value of current and future 
labour income). The optimal share 0 depends on total wealth as well, but also on interest 
rates, the degree of risk-aversion and the level 3 of the jump of the risky price in (11.1.13). 
Hence, it is possible to work with complex stochastic models that allow to analyse many 
interesting real-world features and nevertheless end up with explicit closed-form solutions. 
Many further examples exist - see ch. 11.6 on “further reading”. 


l-o T2 Tə — rı 


e Finding value functions for special cases 


As we have just seen, value functions and closed-form solutions can be found for 
some models which have “nice features”. For a much larger class of models - which are 
then standard models - closed-form solutions cannot be found for general parameter sets. 
Economists then either go for numerical solutions, which preserves a certain generality 
as in principle the properties of the model can be analyzed for all parameter values, or 
they restrict the parameter set in a useful way. Useful means that with some parame- 
ter restriction, value functions can be found again and closed-form solutions are again 
possible. 


11.1.6 Expected growth 


Let us now try to understand the impact of uncertainty on expected growth. In order to 
compute expected growth of consumption from realized growth rates (11.1.12), we rewrite 
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this equation as 


c(t) 
č (t) 


Expressing it in its integral version as in (10.5.15), we obtain for T > t, 


o [e (T) — c (t) = f {r@-p+a la+s) (2) -joas 


+a f (s) = e(8)} das). 


odet) = fr) -o afa | a) ( ) =a] fee ar tote clo) de 


Applying the expectations operator, given knowledge in t, yields 


oeus im f froot fa+s) (62) -joas 


5) 
+B f {6 (s) —e(s)} dq (s). 


Using again the martingale result from ch. 10.5.3 as already in (10.5.16), i.e. the expression 
in (10.5.7), we replace Æ; ff {č (s) — c (s)} dq (8) by AE, fi {č (s) — e(s)} ds, ie. 


eee E ne; f froot 0+8) (3) -jeas 


+ AE; ig {é(s) —c(s)} ds. 


Differentiating with respect to time 7 yields 


1 oO 
dE,c(r) /dr = zE! [f (7) -—p+A jo +8) (2) — 1 } e(r) bagi {é(r) —c(r)}. 
Let us now put into the perspective of time 7, i.e. let’s move time from t to 7 and let’s 
ask what expected growth of consumption is. This shift in time means formally that our 


expectations operator becomes FE, and we obtain 


ae _ Le, fri) =p ala + b) (G3) -ira {99 - } 


In this step we used the fact that due to this shift in time, c(7) is now known and we can 
pull it out of the expectations operator and divide by it. Combining brackets yields 


eee = tE, fro) -p+À jo + B) ee J ie E- 7 |) 
= 1r, fr -pafa b) ($5) | a siel 
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11.2 Matching on the labour market: where value 
functions come from 


Value functions are widely used in matching models. Examples are unemployment with 
frictions models of the Mortensen-Pissarides type or shirking models of unemployment a 
la Shapiro and Stiglitz (1984). These value functions can be understood very easily on an 
intuitive level, but they really come from a maximization problem of households. In order 
to understand when value functions as the ones used in the just-mentioned examples can 
be used (e.g. under the assumption of no saving, or being in a steady state), we now derive 
value functions in a general way and then derive special cases used in the literature. 


11.2.1 A household 


Let wealth a of a household evolve according to 
da = {ra + z — c} dt. 


Wealth increases per unit of time dt by the amount da which depends on current savings 
ra + z -— c. Labour income is denoted by z which includes income w when employed and 
unemployment benefits b when unemployed, z = w, b. Labour income follows a stochastic 
Poisson differential equation as there is job creation and job destruction. In addition, we 
assume technological progress that implies a constant growth rate g of labour income. 
Hence, we can write 


dz = gzdt — Adq, + Ado, 


where A = w — b. Job destruction takes place at an exogenous, state-dependent, arrival 
rate s (z) . The corresponding Poisson process counts how often our household moved from 
employment into unemployment which is q,. Job creation takes place at an exogenous 
rate A(z) which is related to the matching function presented in (5.6.17). The Poisson 
process related to the matching process is denoted by qp. It counts how often a household 
leaves its “b-status”, i.e. how often a job is found. As an individual cannot lose his job 
when he does not have one and as finding a job makes (in this setup) no sense for someone 
who has a job, both arrival rates are state dependent. As an example, when an individual 
is employed, À (w) = 0, when he is unemployed, s (b) = 0. 


© S| oœ 


A(z) 0 
( s 


Table 11.2.1 State dependent arrival rates 


Let the individual maximize expected utility Æ, f° e-?!"u (c(T)) dr, where instan- 
“=! with o > 0. 


l-o 


taneous utility is of the CES type, u (c) = £ 
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11.2.2 The Bellman equation and value functions 


The state space is described by a and z. The Bellman equation has the same structure 
as in (11.1.4). The adjustments that need to be made here follow from the fact that we 
have two state variables instead of one. Hence, the basic structure from (11.1.4) adopted 
to our problem reads pV (a, z) = max, {u (c) + 4E;dV (a, z)}. The change of V (a, z) is, 
given the evolution of a and z from above and the CVF from (10.2.11), 


dV (a, z) = {Va [ra + z — c] + V.gz} dt 
+ {V (a,z — A) — V (a, z)} dqu + {V (a,z + A) — V (a, z)} do. 


Forming expectations, remembering that F,dq, = s (z) dt and Eydq, = (z) dt, and “di- 
viding” by dt gives the Bellman equation 


ov (a2) = mae Wo) + Valrah ed 4 Vag 
= MX 4.5 (z) [V (a,z — A) = V (a, 2)] +A (2) [V (a, z +A) — V (a,2) 
(11.2.1) 
The value functions in the matching literature are all special cases of this general Bellman 
equation. 

Denote by U = V (a,b) the expected present value of (optimal behaviour of a worker) 
being unemployed (as in Pissarides, 2000, ch. 1.3) and by W = V (a,w), the expected 
present value of being employed. As the probability of losing a job for an unemployed 
worker is zero, s (b) = 0, and A (b) = À, the Bellman equation (11.2.1) reads 


pU = max {u (c) + Ua [ra + b — c] + Uzgb + A [W — U]}, 


where we used that W = V (a,b + A). When we assume that agents behave optimally, i.e. 
we replace control variables by their optimal values, we obtain the maximized Bellman 
equation, 


pU = u (c) + Ua [ra +b — c] + Uzgb + à [W — U]. 


When we now assume that households can not save, i.e. c = ra + b, and that there is 
no technological progress, g = 0, we obtain 


pU =u(ra+b)+A[W — U]. 
Assuming further that households are risk-neutral, i.e. u (c) = c, and that they have no 
capital income, i.e. a = 0, consumption is identical to unemployment benefits c = b. If 


the interest rate equals the time preference rate, we obtain eq. (1.10) in Pissarides (2000), 


rU =b+AÀ[W —U]. 
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11.3 Intertemporal utility maximization under Brown- 
ian motion 


11.3.1 The setup 


Consider an individual whose budget constraint is given by 
da = {ra + w — pc} dt + Badz. 


The notation is as always, uncertainty stems from Brownian motion z. The individual 
maximizes a utility function as given in (11.1.1), U (t) = E; fe"! u(e(r)) dr. The 
value function is defined by V (a) = maxy.,,); U (t) subject to the constraint. We again 
follow the three step scheme for dynamic programming. 


11.3.2 Solving by dynamic programming 


e DP1: Bellman equation and first-order conditions 


The Bellman equation is given for Brownian motion by (11.1.4) as well. When a 
maximization problem other than one where (11.1.4) is suitable is to be formulated and 
solved, five adjustments are, in principle, possible for the Bellman equation. First, the 
discount factor p might be given by some other factor - for example the interest rate r 
when the present value of some firm is maximized. Second, the number of arguments 
of the value function needs to be adjusted to the number of state variables. Third, the 
number of control variables depends on the problem that is to be solved and, fourth, the 
instantaneous utility function is replaced by what is found in the objective function - 
which might be, for example, instantaneous profits. Finally and obviously, the differential 
dV (.) needs to be computed according to the rules that are appropriate for the stochastic 
processes which drive the state variables. 

As the differential of the value function, following Ito’s Lemma in (10.2.3), is given by 


dV (a) = fv (a) [ra + w — pe] + sv" (a) a? dt + V' (a) Badz, 


forming expectations F, and dividing by dt yields the Bellman equation for our specific 
problem 


1 
pv (a) = max} u(e(t)) + V' (a) ra +w = pe] + 3V” (a) 8a} 
and the first-order condition is 


u (c(t)) = V' (a) p(t). (11.3.1) 
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e DP2: Evolution of the costate variable 


(i) The derivative of the maximized Bellman equation with respect to the state variable 
gives (using the envelope theorem) an equation describing the evolution of the costate 
variable, 


1 
pV’ = V" [ra + w -— pe] + V'r + zV” Ba +V" Ba 


1 
(p t r) V’ = y" [ra Lay = pc] L 5V "Ba + V" Ba. (11.3.2) 


Not surprisingly, given that the Bellman equation already contains the second derivative 
of the value function, the derivative of the maximized Bellman equation contains its third 
derivative V”. 

(ii) Computing the differential of the shadow price of wealth V’ (a) gives, using Ito’s 
Lemma (10.2.3), 


1 
dV' = V"da + zV” B adt 
1 
= V" [ra + w — pc] dt + zV” Padt + V” Badz, 
and inserting into the partial derivative (11.3.2) of the maximized Bellman equation yields 
dV’ = (p — r) V'dt — V" B’adt + V" Badz 
= {(p — r) V' — V" B’a} dt + V" Badz. (11.3.3) 


As always at the end of DP2, we have a differential equation (or difference equation in 
discrete time) which determines the evolution of V’ (a) , the shadow price of wealth. 


e DP3: Inserting first-order conditions 


Assuming that the evolution of aggregate prices is independent of the evolution of 
the marginal value of wealth, we can write the first-order condition for consumption in 
(11.3.1) as du’ (c) = pdV' + V'dp. This follows, for example, from Ito’s Lemma (10.2.6) 
with ppv = 0. Using (11.3.3) to replace dV’, we obtain 


du' (c) =p [{(p — r) V' — V"B? a} dt + V"Badz| + V'dp 
= {(p—1r)u' (c) —u" (ec) ¢ (a) Ba} dt +u” (c) d (a) Badz + u’ (c) dp/p, (11.3.4) 


where the second equality uses the first-order condition u’ (c(t)) = V'p (t) to replace V’ 
and the partial derivative of this first-order condition with respect to assets, u” (c) d (a) = 
V"p, to replace V”. 

When comparing this with the expression in (11.1.9) where uncertainty stems from a 
Poisson process, we see two common features: First, both Keynes-Ramsey rules have a 
stochastic term, the dz-term here and the dq-term in the Poisson case. Second, uncertainty 
affects the trend term for consumption in both terms. Here, this term contains the second 
derivative of the instantaneous utility function and c (a), in the Poisson case, we have 
the č terms. The additional dp-term here stems from the assumption that prices are not 
constant. Such a term would also be visible in the Poisson case with flexible prices. 
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11.3.3 The Keynes-Ramsey rule 


Just as in the Poisson case, we want a rule for the evolution of consumption here as well. 
We again define an inverse function and end up in the general case with 


Se = [eiB Duaja aas 
ae OF Badz — ae 
ul (e) ° p 


With a CRRA utility function, we can replace the first, second and third derivative of 
u (c) and find the corresponding rule 


o— = fr- p-0%8a | lojo + 1] [toa] a (11.3.6) 


11.4 Capital asset pricing 


Let us again consider a typical CAP problem. This follows and extends Merton (1990, 
ch. 15; 1973). The presentation is in a simplified way. 


11.4.1 The setup 


The basic structure of the setup is identical as before. There is an objective function and 
a constraint. The objective function captures the preferences of our agent and is described 
by a utility function as in (11.1.1). The constraint is given by a budget constraint which 
will now be derived, following the principles of ch. 10.3.2. 

Wealth a of households consist of a portfolio of assets i 


23 N 
a= X; Pini, 


where the price of an asset is denoted by p; and the number of shares held, by n;. The 
total number of assets is given by N. Let us assume that the price of an asset follows 
geometric Brownian motion, 

dpi 

Pi L adt + oidzi, (11.4.1) 
Pi 
where each price is driven by its own drift parameter a; and its own variance parameter g;. 
Uncertainty results from Brownian motion z; which is also specific for each asset. These 
parameters are exogenously given to the household but would in a general equilibrium 
setting be determined by properties of, for example, technologies, preferences and other 
parameters of the economy. 
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Households can buy or sell assets by using a share y; of their savings, 


1 
dn; = —x; {OM Tins w= ch dt. (11.4.2) 
Pi 


When savings are positive and a share x, is used for asset 7, the number of stocks held in 7 
increases. When savings are negative and y; is positive, the number of stocks 7 decreases. 

The change in the households wealth is given by da = Ed (pini). The wealth held 
in one asset changes according to 


d (pini) = pidn; + nidp; = x; {E amini + w — c} dt + nidpi. 


The first equality uses Ito’s Lemma from (10.2.6), taking into account that second deriv- 
atives of F (.) = pn; are zero and that dn; in (11.4.2) is deterministic and there- 
fore dp;dn; = 0. Using the pricing rule (11.4.1) and the fact that shares add to unity, 
E` x; = 1, the budget constraint of a household therefore reads 


da = {ON ring + w — c} dt + OL nip; [aidt + oidz] 


= {Bea tnp Epes w= e} dt + OM nipoidz; 
Pi 
= bee E + a +w- e} dt + YX aoidz;. 
Pi 


Now define 0; as always as the share of wealth held in asset i, 0; = a;/a. Then, by 
definition, a = X; 0;a and shares add up to unity, U_,0; = 1. We rewrite this for later 
purposes as 

On =1-— ENO; (11.4.3) 


Define further the interest rate for asset i and the interest rate of the market portfolio by 


r= ees a, T= ÐA biri. (11.4.4) 


This gives us the budget constraint, 


da = {aX iri +w- ch dt + aX bicidzi 
= {ra + w — c} dt + ad ,0;0 dz}. (11.4.5) 


11.4.2 Optimal behaviour 


Let us now consider an agent who behaves optimally when choosing her portfolio and in 
making her consumption-saving decision. We will not go through all the steps to derive 
a Keynes-Ramsey rule as asset pricing requires only the Bellman equation and first-order 
conditions. 
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e The Bellman equation 


The Bellman equation is given by (11.1.4), i.e. pV (a) = max.) 9, {u (c (t)) + #EdV (a)}. 
Hence, we need again the expected change of the value of one unit of wealth. With one 
state variable, we simply apply Ito’s Lemma from (10.2.1) and find 


1 1 1 
— FdV (a) = He (v (a) da + ee (a) aa?) . (11.4.6) 
In a first step required to obtain the explicit version of the Bellman equation, we com- 
pute the square of da. It is given, taking (10.2.2) into account, by [da]? =Q? [aM A:oidz;| A 
When we compute the square of the sum, the expression for the product of Brownian 
motions in (10.2.5) becomes important as correlation coefficients need to be taken into 
consideration. Denoting the covariances by oij = 0;0;p;;, we get 
[da]? = a? [O,01dz + Ao00d zo Plesk OnOn den] 
E a? [Oiojdt + 0101020 2P,.dt ae 93020101 pdt + 6303dt Feat | 
= a OF De 0:0 adt. (11.4.7) 


Now rewrite the sum in (11.4.7) as follows 


pe 19 O05; = 10,804 + EN OONoiN 
= ETEN 0i + DL G0 non +E ONnGj0N; 
= DA DAT OOO T DATTOONCGiN + DICI ONOGONj + Oran 


= EOD: + 203409 noi + ONON 


As the second term, using (11.4.3), can be written as U45'0,0noin = [1 — Dao] SATTON, 
our (da)? reads 


(da)? = a? {DL ENT Oboy + 2 [1 = ea) DAT Oiciw + [1 = sto.) oà) dt 
= az bn 0% = 20in) + 2DAN + [1 = paar oh) dt. (11.4.8) 


The second preliminary step for obtaining the Bellman equation uses (11.4.3) again 
and expresses the interest rate from (11.4.4) as a sum of the interest rate of asset N (which 
could but does not need to be a riskless asset) and weighted excess returns r; — ry, 


r= E biri + (1 EÈ 0) ry = rw + ERG O [ri — ry]. (11.4.9) 


The Bellman equation with (11.4.8) and (11.4.9) now finally reads 


‘ NT Oiiri —rn])atw—e Vv" (a al’ 
pV (a) = max 4 ule) +V (0) [lew + EM lri rea +w = d + 5V” (0 dah, 


where (da)? should be thought of representing (11.4.8). 
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e First-order conditions 


The first-order conditions are the first-order condition for consumption, 


u' (c) =V"(a), 
and the first-order condition for assets. The first-order condition for consumption has the 


well-known form. 
To compute first-order conditions for shares 6;, we compute d {.} /d0; for (11.4.8), 


ath } Z a 0; ilij = 20in) + ENT Oli = 20;n) + 20iN —2 [1 = eal Oo 
=2 Ears 6; (Gi; — 20;n) + OjN — [1 se 2a 10; i| on} 
= 2 {XL bloy — oin) + [1 — D596] (own — on) } 


Hence, the derivative of the Bellman equation with respect to 0; is with (11.4.9) and 
(11.4.10) 


V'alr; — ryn] + SV "20D bilo, —owl=0e 
y" N 

Ti -—TN = Tp Ojai [oi = Gin] : (11.4.11) 
The interpretation of this optimality rule should take into account that we assumed that 
an interior solution exists. This condition, therefore, says that agents are indifferent 
between the current portfolio and a marginal increase in a share 6; if the difference in 
instantaneous returns, r; — ry, is compensated by the covariances of of assets i and N. 
Remember that from (11.4.4), instantaneous returns are certain at each instant. 


11.4.3 Capital asset pricing 


Given optimal behaviour of agents, we now derive the well-known capital asset pricing 
equation. Start by assuming that asset N is riskless, i.e. oy = 0 in (11.4.1). This implies 
that it has a variance of zero and therefore 2 covariance on; with any other asset of zero 
as well, oy; = 0 Vj. Define further y = aX, the covariance of asset i vith the market 
portfolio as Cim = =p ,9;0;:;, the variance of the market portfolio as o? = oe 10705 
and the return of the markel portfolio as r = XÑ; 0;r; as in (11.4.4). 

We are now only few steps away from the CAP equation. Using the definition of y 
and oj allows to rewrite the first-order condition for shares (11.4.11) as 


Tri —TN = YOim- (11.4.12) 


Multiplying this first-order condition by the share 6; gives 0; |r; — rn] = 9;yoim. Summing 
up all assets, i.e. applying oe X; to both sides, and using the above definitions yields 


r—TNn = og: 
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Dividing this expression by version (11.4.12) of the first-order condition yields the capital 
asset pricing equation, 


The ratio cim /0%, is what is usually called the 6-factor. 


11.5 Natural volatility II 


Before this book comes to an end, the discussion of natural volatility models in ch. 8.4 is 
completed in this section. We will present a simplified version of those models that appear 
in the literature which are presented in stochastic continuous time setups. The usefulness 
of Poisson processes will become clear here. Again, more background is available on 
http: //www.waelde.com/nv.html. 


11.5.1 An real business cycle model 


This section presents the simplest general equilibrium setup that allows to study fluctua- 
tions stemming from occasional jumps in technology. The basic belief is that economically 
relevant changes in technologies are rare and occur every 5-8 years. Jumps in technology 
means that the technological level, as captured by the TFP level, increases. Growth cycles 
therefore result without any negative TFP shocks. 


e Technologies 
The economy produces a final good by using a Cobb-Douglas technology 
Y = K* (AL). (11.5.1) 


Total factor productivity is modelled as labour augmenting labour productivity. While 
this is of no major economic importance given the Cobb-Douglas structure, it simplifies 
the notation below. Labour productivity follows a geometric Poisson process with drift 


dA/A = gdt + ydq, (11.5.2) 


where g and y are positive constants and À is the exogenous arrival rate of the Poisson 
process q. We know from (10.5.17) in ch. 10.5.4 that the growth rate of the expected 
value of A is given by g + Av. 

The final good can be used for consumption and investment, Y = C + I, which implies 
that the prices of all these goods are identical. Choosing Y as the numéraire good, the 
price is one for all these goods. Investment increases the stock of production units K if 
investment is larger than depreciation, captured by a constant depreciation rate ô, 


dK = (Y —C — ôK) dt. (11.5.3) 


There are firms who maximize instantaneous profits. They do not bear any risk and 
pay factors r and w, marginal productivities of capital and labour. 
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e Households 


Households maximize their objective function 


U (t) = E f el"! (e (r)) dr 


t 


by choosing the consumption path {c(7)}. Instantaneous utility can be specified by 
l-o _ 1 
u (c) = m 


cg 

Wealth of households consists of shares in firms which are denoted by k. This wealth 
changes in a deterministic way (we do not derive it here but it could be done following 
the steps in ch. 10.3.2), despite the presence of TFP uncertainty. This is due to two facts: 
First, wealth is measured in units of physical capital, i.e. summing k over all households 
gives K. As the price of one unit of K equals the price of one unit of the output good 
and the latter was chosen as numéraire, the price of one unit of wealth is non-stochastic. 
This differs from (10.3.8) where the price jumps when q jumps. Second, a jump in q does 
not affect k directly. This could be the case when new technologies make part of the old 
capital stock obsolete. Hence, the constraint of households is a budget constraint which 
reads 


(11.5.4) 


dk = (rk + w — c) dt. (11.5.5) 


The interest rate is given by the difference between the marginal product of capital and 
the depreciation rate, r = OY/OK — ô. 


e Optimal behaviour 


When computing optimal consumption levels, households take the capital stock k and 
the TFP level A as their state variables into account. This setup is therefore similar to 
the deterministic two-state maximization problem in ch. 6.3. Going through similar steps 
(concerning, for example, the substituting of cross derivatives in step DP2) and taking 
the specific aspects of this stochastic framework into account, yields following optimal 
consumption (see exercise 8) 


o= {r—p+a[(e)"-a] pate f- 1} (11.5.6) 


Despite the deterministic constraint (11.5.5) and due to TFP jumps in (11.5.2), consump- 
tion jumps as well: a dq—term shows up in this expression and marginal utility levels 
before (c77) and after (¢~7) the jump, using the notation from (10.1.7) appear as well. 
Marginal utilities appear in the deterministic part of this rule due to precautionary saving 
considerations. The reason for the jump is straightforward: whenever there is a discrete 
increase in the TFP level, the interest rate and wages jump. Hence, returns for savings or 
households change and the household adjusts its consumption level. This is in principle 
identical to the behaviour in the deterministic case as illustrated in fig. 5.6.1 in ch. 5.6.1. 
(Undertaking this here for this stochastic case would be very useful.) 
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e General equilibrium 


We are now in a position to start thinking about the evolution of the economy as 
a whole. It is described by a system in three equations. Labour productivity follows 
(11.5.2). The capital stock follows 


dK = (K° (AL) ° — C — ôK) dt 


from (11.5.1) and (11.5.3). Aggregate consumption follows 


dC AL eo? oe Č 
= — —6- Al{ +=) —1| >d —-—I1>od 
ep lE arte} 
from aggregating over households using (11.5.6). Individual consumption c is replaced by 
aggregate consumption C and the interest rate is expressed by marginal productivity of 
capital minus depreciation rate. This system looks fairly similar to deterministic models, 


the only substantial difference lies in the dq term and the post-jump consumption levels 


C. 
e Equilibrium properties for a certain parameter set 


The simplest way to get an intuition about how the economy evolves consists in looking 
at an example, i.e. by looking at a solution of the above system that holds for a certain 
parameter set. We choose as example the parameter set for which the saving rate is 
constant and given by s = 1—o~'. The parameter set for which C = (1 — s) Y is optimal, 
is given by ø = (p+d—A [1+] -— 1]) / (ad - (1 —a)g). As we need o > 1 fora 
meaningful saving rate, the intertemporal elasticity of substitution 7~! is smaller than one. 
For a derivation of this result, see the section on “Closed-form solutions with parameter 
restrictions” in “further reading”. 

The dynamics of capital and consumption can then be best analyzed by looking at 
auxiliary variables, in this case capital per effective worker, K = K /A. This auxiliary 
variable is needed to remove the trend out of capital K. The original capital stock grows 
without bound, the auxiliary variable has a finite range. Using auxiliary variables of this 
type has a long tradition in growth models as detrending is a common requirement for 
an informative analysis. The evolution of this auxiliary variable is given by (applying the 
appropriate CVF) 


dk = {8,K°L'* — BR} dt — B,Kdg, (11.5.7) 


where (, are functions of preference and technology parameters of the model. One can 
show that 0 < $3 < 1. The evolution of this capital stock per effective worker can be 
illustrated by using a figure which is similar to those used to explain the Solow growth 
model. The following figure shows the evolution of the capital stock per worker on the 
left and the evolution of GDP on the right. 


290 Chapter 11. Infinite horizon models 


time 


Figure 11.5.1 Cyclical growth 


Assume the initial capital stock K is given by Ko. Assume also, for the time being, that 
there is no technology jump, i.e. dg = 0. The capital stock K then increases smoothly over 
time and approaches a temporary steady state K* which follows from (11.5.7) with dq = 0 
and dK = 0. This temporary steady state is given by K* = (B 1-2 B) 079 and has 
properties in perfect analogy to the steady state in the Solow model. When a technology 
jump occurs, the capital stock per effective worker diminishes as the denominator in 
K=K /A increases but the capital stock in the numerator in the instant of the jump 
does not change. The capital stock per effective worker is “thrown back”, as indicated by 
the arrow in the left panel above. After that, the capital stock K again approaches the 
temporary steady state K*. 

The right panel of the above figure shows what these technology jumps do to GDP. As 
long as there is no technology jump, GDP approaches an upper limit Y% which is specific to 
the technology level q. A technology jump increases GDP Y} instantaneously as TFP goes 
up. (This increase could be very small, depending on what share of production units enjoy 
an increase in TFP.) The more important increase, however, results from the the shift 
in the upper limit from Y% to Y;.,. Capital accumulation following the technology jump 
increases TFP which now approaches the new upper limit. This process of endogenous 
growth cycles continues ad infinitum. 


11.5.2 A natural volatility model 


The above analysis can be extended to allow for endogenous technology jumps. As in 
the discrete time version of this setup, the probability that a technology jump occurs is 
made a function of resources R invested into R&D. In contrast to (8.4.3), however, it is 
the arrival rate and not the probability itself which is a function of R, 


AAR) & 
This is a requirement of continuous time setups and is builds on a long tradition in 


continuous time models with Poisson processes. The resource constraint of the economy 
is then extended accordingly to 


dK =(Y -C—R-6K)dt 
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by including the resources R. 

A model of this type can then be solved as before. Either one considers special pa- 
rameter values and obtains closed-form solutions or one performs a numerical analysis 
(see below). The qualitative properties of cyclical growth are identical to the ones pre- 
sented before in fig. 11.5.1. The crucial economic difference consists in the fact that the 
frequency of technology jumps, i.e. the average length of a business cycle, depend on de- 
cisions of households. If households find it profitable to shift a larger amount of resources 
into R&D, business cycles will be shorter. Not only the long-run growth rate, but also 
short-run fluctuations are influenced by fundamental parameters of the model as also by 
government policy. All these questions are analyzed in detail in the “natural volatility” 
literature. 


11.5.3 A numerical approach 


It is helpful for a numerical solution to have a model description in stationary variables. 
To this end, define auxiliary variables K = K/A and C = C/A. The one for capital is 
the same as was used in (11.5.7), the one for consumption is new as we will now not work 
with a closed-form solution. Let us look at a situation with exogenous arrival rates, i.e. at 
the RBC model of above, to illustrate the basic approach for a numerical solution. When 
computing the dynamics of these variables (see “further reading” for references), we find 


A iza A (o ps 
$- fa(4) —d-—p-—ogt+A ( o :) wi] hats} Saha, 
© 7 ie i (11.5.8) 


dK = {Y =a(6 Py) k= ô) dt — Raq. (11.5.9) 
Ley 
When we look at these equations, they “almost” look like ordinary differential equa- 


tions. The only difficulty is contained in the term C. To understand the solution proce- 
dure, think of the saddle-path trajectory in the optimal growth model - see fig. 5.6.2 in 
ch. 5.6.3. Given the transformation undertaken here, the solution of this system is given 


by a policy function Ĉ (£ ) which is not a function of the technological level A. The ob- 
jective of the solution therefore consists of finding this C (£ ) , a saddle path in analogy 


to the one in fig. 5.6.2. The term C stands for the consumption level after the jump. As 
the functional relationship is independent of A and thereby the number of jumps, we can 
write C =C (4k ) , Le. it is the consumption level at the capital stock ik after a 


jump. As y > 0, a jump implies a reduction in the auxiliary, i.e. technology-transformed, 
capital stock K. 
The “trick” in solving this system now consists of providing not only two initial con- 


ditions but one initial condition for capital an initial path for consumption C’ (£ | This 


initial path is then treated as an exogenous variable in the denominator of (11.5.8) and 
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the above differential equations have been transformed into a system of ordinary differ- 
ential equations! Letting the initial paths start at the origin and making them linear, the 
question then simply consists of finding the right slope such that the solution of the above 
system identifies a saddle path and steady state. For more details and an implementation, 
see “Numerical solution” in “further reading”. 


11.6 Further reading and exercises 
e Mathematical background on dynamic programming 


There are many, and in most cases much more technical, presentations of dynamic 
programming in continuous time under uncertainty. A classic mathematical reference 
is Gihman and Skorohod (1972) and a widely-used mathematical textbook is Øksendal 
(1998); see also Protter (1995). These books are probably useful only for those wishing 
to work on the theory of optimization and not on applications of optimization methods. 
Kushner (1967) and Dempster (1991) have a special focus on Poisson processes. Op- 
timization with unbounded utility functions by dynamic programming was studied by 
Sennewald (2007). 

With SDEs we need boundary conditions as well. In the infinite horizon case, we would 
need a transversality condition (TVC). See Smith (1996) for a discussion of a TVC in a 
setup with Epstein-Zin preferences. Sennewald (2007) has TVCs for Poisson uncertainty. 


e Applications 


Books in finance that use dynamic programming methods include Duffie (1988, 2001) 
and Björk (2004). Stochastic optimization for Brownian motion is also covered nicely in 
Chang (2004). 

A maximization problem of the type presented in ch. 11.1 was first analyzed in Wilde 
(1999, 2008). This chapter combines these two papers. These two papers were also jointly 
used in the Keynes-Ramsey rule appendix to Wälde (2005). It is also used in Posch and 
Wälde (2006), Sennewald and Walde (2006) and elsewhere. 

Optimal control in stochastic continuous time setups is used in many applications. 
Examples include issues in international macro (Obstfeld, 1994, Turnovsky, 1997, 2000), 
international finance and debt crises (Stein, 2006) and also the analysis of the permanent- 
income hypothesis (Wang, 2006) or of the wealth distribution hypothesis (Wang, 2007), 
and many others. A firm maximization problem with risk-neutrality where R&D increases 
quality of goods, modelled by a stochastic differential equation with Poisson uncertainty, 
is presented and solved by Dinopoulos and Thompson (1998, sect. 2.3). 

The Keynes-Ramsey rule in (11.3.4) was derived in a more or less complex framework 
by Breeden (1986) in a synthesis of his consumption based capital asset pricing model, 
Cox, Ingersoll and Ross (1985) in their continuous time capital asset pricing model, or 
Turnovsky (2000) in his textbook. Cf. also Obstfeld (1994). 
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There are various recent papers which use continuous time methods under uncertainty. 
For examples from finance and monetary economics, see DeMarzo and Urošević (2006), 
Gabaix et al. (2006), Maenhout (2006), Piazzesi (2005), examples from risk theory and 
learning include Kyle et al. (2006) and Keller and Rady (1999), for industrial organiza- 
tion see, for example, Murto (2004). In spatial economics there is, for example, Gabaix 
(1999), the behaviour of households in the presence of durable consumption goods is an- 
alyzed by Bertola et al. (2005), R&D dynamics are investigated by Bloom (2007) and 
mismatch and exit rates in labour economics are analyzed by Shimer (2007,2008). The 
effect of technological progress on unemployment is analyzed by Prat (2007). The real 
options approach to investment or hiring under uncertainty is another larger area. See, 
for example, Bentolila and Bertola (1990) or Guo et al. (2005). Further references to 
papers that use Poisson processes can be found in ch. 10.6. 


e Closed form solutions 


Closed-form solutions and analytical expressions for value functions have been derived 
by many authors. This approach was pioneered by Merton (1969, 1971) for Brownian 
motion. Chang (2004) devotes an entire chapter (ch. 5) to closed-form solutions for 
Brownian motion. For setups with Poisson-uncertainty, Dinopoulos and Thompson (1998, 
sect. 2.3), Walde (1999b) or Sennewald and Walde (2006, ch. 3.4) derive closed-form 
solutions. An overview is provided by Wilde (2011). 

Closed-form solutions for Levy processes are available, for example, from Aase (1984), 
Framstad, Øksendal and Sulem (2001) and in the textbook by Øksendal and Sulem (2005). 


e Closed-form solutions with parameter restrictions 


Sometimes, restricting the parameter set of the economy in some intelligent way allows 
to provide closed-form solutions for very general models. These solutions provide insights 
which cannot be obtained that easily by numerical analysis. Early examples are Long 
and Plosser (1983) and Benhabib and Rustichini (1994) who obtain closed-form solutions 
for a discrete-time stochastic setup. In deterministic, continuous time, Xie (1991, 1994) 
and Barro, Mankiw and Sala-i-Martin (1995) use this approach as well. Wälde (2005) 
and Wilde and Posch (2006) derive closed-form solutions for business cycle models with 
Poisson uncertainty. Sennewald and Wälde (2006) study an investment and finance prob- 
lem. The example in section 11.5.1 is taken from Schlegel (2004). The most detailed 
step-by-step presentation of the solution technique is in the Referees’ appendix to Wälde 
(2005). 


e Natural volatility 
The expression “natural volatility” represents a certain view about why almost any 


economic time series exhibits fluctuations. Natural volatility says that fluctuations are 
natural, they are intrinsic to any growing economy. An economy that grows is also an 
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economy that fluctuates. Growth and fluctuations are “two sides of the same coin”, they 
have the same origin: new technologies. 

Published papers in this literature are (in sequential and alphabetical order) Fan 
(1995), Bental and Peled (1996), Freeman, Hong and Peled (1999), Matsuyama (1999, 
2001), Wälde (1999, 2002, 2005), Li (2001) Francois and Lloyd-Ellis (2003,2008), Maliar 
and Maliar (2004) and Phillips and Wrase (2006). 


e Numerical solution 
The numerical solution was analyzed and implemented by Schlegel (2004). 
e Matching and saving 


Ch. 11.2 shows where value functions in the matching literature come from. This 
chapter uses a setup where uncertainty in labour income is combined with saving. It 
thereby presents the typical setup of the saving and matching literature. The setup used 
here was explored in more detail by Bayer and Walde (2010a, b) and Bayer et al. (2010, 


The matching and saving literature in general builds on incomplete market models 
where households can insure against income risk by saving (Huggett, 1993; Aiyagari, 1994; 
Huggett and Ospina, 2001; Marcet et al., 2007). First analyses of matching and saving 
include Andolfatto (1996) and Merz (1995) where individuals are fully insured against 
labour income risk as labour income is pooled in large families. Papers which exploit 
the advantage of CARA utility functions include Acemoglu and Shimer (1999), Hassler 
et al. (2005), Shimer and Werning, (2007, 2008) and Hassler and Rodriguez Mora, 1999, 
2008). Their closed-form solutions for the consumption-saving decision cannot always rule 
out negative consumption levels for poor households. Bils et al. (2007, 2009), Nakajima 
(2008) and Krusell et al. (2010) work with a CRRA utility function in discrete time. 


11.6. Further reading and exercises 295 


Exercises Chapter 11 
Applied Intertemporal Optimization 


Dynamic Programming in continuous time 
under uncertainty 


1. Optimal saving under Poisson uncertainty with two state variables 
Consider the objective function 


and the budget constraint 
da (t) = {ra (t) + w — p (t) c (t)} dt + Ba (t) dq (t), 


where r and w are constant interest and wage rates, q (t) is a Poisson process with 
an exogenous arrival rate \ and 8 is a constant as well. Letting g and o denote 
constants, assume that the price p(t) of the consumption good follows 


dp (t) = p (t) [gdt + adq (t)]. 
(a) Derive a rule which optimally describes the evolution of consumption. Deriving 
this rule in the form of marginal utility, i.e. du’ (c (t)) is sufficient. 


(b) Derive a rule for consumption, i.e. de(t) =... 


(c) Derive a rule for optimal consumption for 8 = 0 or o = 0. 


2. Optimal saving under Brownian motion 
Derive the Keynes-Ramsey rules in (11.3.5) and (11.3.6), starting from the rule for 
marginal utility in (11.3.4). 


3. Adjustment cost 
Consider a firm that maximizes its present value defined by II = F; 1 elt—!q (7) dr. 
The firm’s profit 7 is given by the difference between revenues and costs, 7 = pr—ci?, 
where output is assumed to be a function of the current capital stock, x = k“. The 
firm’s control variable is investment 7 that determines its capital stock, 


dk = (i — ôk) dt. 
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The firm operates in an uncertain environment. Output prices and costs for invest- 
ment evolve according to 


dp/p = dt +Opd%,  dc/c = a,dt + o-dze, 
where zp and z, are two independent stochastic processes. 


(a) Assume z, and z, are two independent Brownian motions. Set a, = op = 0, 
such that the price p is constant. What is the optimal investment behaviour 
of the firm? 


(b) Consider the alternative case where costs c are constant but prices p follow the 
above SDE. How much would the firm now invest? 


(c) Provide an answer to the question in a) when z, is a Poisson process. 


(d) Provide an answer to the question in b) when z, is a Poisson process. 


4. Firm specific technological progress 
Consider a firm facing a demand function with price elasticity €, £x = @p-*, where 
p is the price and ® is a constant. Let the firm’s technology be given by x = afl 
where a > 1. The firm can improve its technology by investing in R&D. R&D is 
modelled by the Poisson process q which jumps with arrival rate A (l4) where l is 
employment in the research department of the firm. The exogenous wage rate the 
firm faces amounts to w. 


(a) What is the optimal static employment level / of this firm for a given techno- 
logical level q? 


(b) Formulate an intertemporal objective function given by the present value of 
the firms profit flows over an infinite time horizon. Continue to assume that 
q is constant and let the firm choose l optimally from this intertemporal per- 
spective. Does the result change with respect to (a)? 


(c) Using the same objective function as in (b), let the firm now determine both / 
and J, optimally. What are the first-order conditions? Give an interpretation 
in words. 


(d) Compute the expected output level for 7 > t, given the optimal employment 
levels (* and l}. In other words, compute Ez (T). Hint: Derive first a stochastic 
differential equation for output x (t). 


5. Budget constraints and optimal saving and finance decisions 
Imagine an economy with two assets, physical capital K (t) and government bonds 
B (t). Let wealth a(t) of households be given by a(t) = v (t) k (t) + b(t) where 
v (t) is the price of one unit of capital, k (t) is the number of stocks and b(t) is the 
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nominal value of government bonds held by the household. Assume the price of 
stocks follows 


dv (t) = av (t) dt + Bu (t) dq (t), 


where a and ĝ are constants and q(t) is a Poisson process with arrival rate A. 


(a) Derive the budget constraint of the household. Use 0 (t) = v (t) k (t) /a (t) as 
the share of wealth held in stocks. 


(b) Derive the budget constraint of the household by assuming that q (t) is Brown- 
ian motion. 


(c) Now let the household live in a world with three assets (in addition to the two 
above, there are assets available on foreign markets). Assume that the budget 
constraint of the household is given by 


da (t) = {r (t) a(t) + w (t) — p (t) c (t)} dt+8,0r (t) a (t) dq (t) +8 30s (t) a (t) day (t), 
where 
r (t) = Ok (t) re + Of (t)r + (1 — 0r (t) — 97 (t)) ro 


is the interest rate depending on weights 0; (t) and constant instantaneous 
interest rates r, ry and rp. Let q (t) and qp (t) be two Poisson processes. Given 
the usual objective function 


U (t) = B, / * e-olHa (e (7)) dr, 


what is the optimal consumption rule? What can be said about optimal shares 
0% and 0%? 


6. Capital asset pricing in ch. 11.4.3 
The covariance of asset 7 with the market portfolio is denoted by cim, the variance 
of the market portfolio is 0%, and the return of the market portfolio is rm = ©,6;7;. 


(a) Show that the covariance of asset i with the market portfolio oim is given by 
EN bjd. 
j=10 4 ij 


(b) Show that EN Oio; m is the variance of the market portfolio o?z. 
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7. Standard and non-standard technologies 
Let the social welfare function of a central planner be given by 


U (t) = E [ e7Pl™—4y (C (r)) dr. 


(a) Consider an economy where the capital stock follows dK = AK°L'~° [udt + odz]— 
(dK +C) dt where dz is the increment of Brownian motion and u and o are 
constants. Derive the Keynes-Ramsey rule for this economy. 


(b) Assume that dY = O [udt + odz] and that © is constant. What is the expected 
level of Y (7) for T > t, ie. Y (T)? 


(c) Consider an economy where the technology is given by Y = AK°L1~° with 
dA = yAdt + 3 Adz, where z is Brownian motion. Let the capital stock follow 
dk = (Y — ôK —C)dt. Derive the Keynes-Ramsey rule for this economy as 
well. 


(d) Is there a parameter constellation under which the Keynes-Ramsey rules are 
identical? 


8. Standard and non-standard technologies II 
Provide answers to the same questions as in "Standard and non-standard technolo- 
gies" but assume that z is a Poisson process with arrival rate A. Compare your result 
to (11.5.6). 


Chapter 12 


Miscellanea, references and index 


The concept of time 


What is time? Without getting into philosophical details, it is useful to be precise 
here about how time is denoted. Time is always denoted by 7. Time can take different 
values. The most important (frequently encountered) one is the value t where t stands for 
today. This is true both for the discrete and for the continuous time context. In a discrete 
time context, t + 1 is then obviously tomorrow or the next period. Another typical value 
of time is tọ, which is usually the point in time for which initial values of some process 
are given. Similarly, T denotes a future point in time where, for example, the planning 
horizon ends or for which some boundary values are given. In most cases, time 7 refers 
to some future point in time in the sense of T > t. 

Given these definitions, how should one present generic transition equations, in most 
examples above budget constraints? Should one use time t as argument or time 7? In 
two-period models, it is most natural to use t for today and t + 1 for tomorrow. This is 
the case in the two-period models of ch. 2, for example, in the budget constraints (2.1.2) 
and (2.1.3). 

This choice becomes less obvious in multi-period models of ch. 3. When the first 
transition equation appears in (3.3.1) and the first dynamic (in contrast to intertemporal) 
budget constraint in (3.4.1), one can make arguments both in favour of t or T as time 
argument. Using rT would be most general: The transition equation is valid for all periods 
in time, hence one should use 7. On the other hand, when the transition equation is valid 
for t as today and as we know that tomorrow will be “today” tomorrow (or: “today turned 
into yesterday tomorrow - morgen ist heute schon gestern”) and that any future point in 
time will be today at some point, we could use t as well. 

In most cases, the choice of t or T as time argument is opportunistic: When a maxi- 
mization method is used where the most natural representation of budget constraints is 
in t, we will use t. This is the case for the transition equation (3.3.1) where the maximiza- 
tion problem is solved by using a Bellman equation. Using t is also most natural when 
presenting the setup of a model as in ch. 3.6. 

If by contrast, the explicit use of many time periods is required in a maximization 
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problem (like when using the Lagrangian with an intertemporal budget constraint as in 
(3.1.3) or with a sequence of dynamic budget constraints in (3.7.2)), budget constraints 
are expressed with time 7 as argument. Using 7 as argument in transition equations is 
also more appropriate for Hamiltonian problems where the current value Hamiltonian is 
used - as throughout this book. See, for example, the budget constraint (5.1.2) in ch. 5.1. 

In the more formal ch. 4 on differential equations, it is also most natural (as this is 
the case in basically all textbooks on differential equations) to represent all differential 
equation with t as argument. When we solve differential equations, we need boundary 
conditions. When we use initial conditions, they will be given by to. If we use a terminal 
condition as boundary condition, we use T > t to denote some future point in time. (We 
could do without to, use 7 as argument and solve for tọ < rT < T. While this would be 
more consistent with the rest of the book, it would be less comparable to more specialized 
books on differential equations. As we believe that the “inconsistency” is not too strong, 
we stick to the more standard mathematical notation in ch. 4). 

Here is now a summary of points in time and generic time 


t a point in time t standing for today (discrete and continuous time) 
generic time (for differential equations in ch. 4 and for model presentations) 
t+1 tomorrow (discrete time) 


i some future point in time, T > t (discrete and continuous time) 
T generic time (for differential equations in some maximization problems 
- e.g. when Hamiltonians are used) 
to a point in time, usually in the past (differential equations) 
T a point in time, usually in the future (differential equations 
and terminal point for planning horizon) 
0 today when t is normalized to zero (only in the examples of ch. 5.5.1) 


More on notation 


The notation is as homogeneous as possible. Here are the general rules but some 
exceptions are possible. 


e Variables in capital, like capital K or consumption C denote aggregate quantities, 
lower-case letters pertain to the household level 


e A function f (.) is always presented by using parentheses (.), where the parenthe- 
ses give the arguments of functions. Brackets [.] always denote a multiplication 
operation 


e A variable x; denotes the value of x in period t in a discrete time setup. A variable 
x(t) denotes its value at a point in time t in a continuous time setup. We will 
stick to this distinction in this textbook both in deterministic and stochastic setups. 
(Note that the mathematical literature on stochastic continuous time processes, i.e. 
what is treated here in part IV, uses x; to express the value of x at a point in time 


t.) 
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e A dot indicates a time derivative, t (t) = dx/dt. 


e A derivative of a function f (x) where z is a scalar and not a vector is abbreviated 
by f'(x) = df (x) /dx. Partial derivatives of a function f (x,y) are denoted by for 
example, fr (x,y) = fe (-) = fe = Of (x, y) /Ox. 


Variables 


Here is a list of variables and abbreviations which shows that some variables have 
multi-tasking abilities, i.e. they have multiple meanings 


e Greek letters 


1/(1+ p) 


YDSRDA DDL WEA 
2. 


pe 
* 


© 


e Latin letters 


output elasticity of capital 

discount factor in discrete time 

preference parameter on first period utility in two-period models 
depreciation rate 

share of wealth held in the risky asset 

share of wealth held in asset i 

instantaneous profits, i.e. profits m, in period t or 7 (t) at point in time t 
time preference rate, correlation coefficient between random variables 
correlation coefficient between two random variables ¿ and j 

see above on “the concept of time” 

the share of savings used for buying stock i 

misprint 


adjustment cost function 


{c} the time path of c from t to infinity, i.e. for all r > t, {c7} = {G, ca, ...} 
{e(T)} the time path of c from t to infinity, i.e. for all r > t 

CVF change of variable formula 

e expenditure e = pc, effort, exponential function 


f (x+y) a function f (.) with argument z + y 
f{x+y] some variable f times x + y 

q (t) Poisson process 

r interest rate 

RBC real business cycle 
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ODE ordinary differential equation 

RV random variable 

SDE stochastic differential equation 

t, T, to see above on “the concept of time” 
TVC transversality condition 

TFP total factor productivity 


u instantaneous utility (see also 7) 

w wk wage rate in period t; factor reward for labour. w# is used to stress difference to w* 
wi factor reward for capital in period t 

ae fixpoint of a difference or differential equation (system) 


z(t) Brownian motion, Wiener process 
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solution 
definition, 15 
vs Euler equation, 17 
vs necessary condition, 17 
solvency condition, see also no-Ponzi game 
condition, 99 
solving a maximization problem, 17 
solving by inserting 
principle, 15 
state variable, 55, 110, 114, 132, 145, 198, 
209, 269, 282 
definition, 50, 56 
steady state, 120, 129, 150 
and limiting distribution, 170 
continuous time, 86, 87 
definition, 65 
discrete time, 30, 31, 67 
temporary, 192, 290 
sticky prices, 212 
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stochastic differential equation, 228, 229, 234, 
236, 242, 245 
solution, 242, 243, 245, 247 
stochastic differential equations 
definition of solution, 242 
substitution, see elasticity of substitution, 
see intertemporal elasticity of sub- 
stitution, see marginal rate of sub- 
stitution 
switch process, 249 


terminal condition, see boundary condition 
time preference rate 

and discount factor, 20 

definition, 20 
transversality condition, 99, 102, 115, 292 


variance 
definition, 165 
of Brownian motion, 226, 256 
of capital stock, 181 
of continuous-time process, 252 
of discrete-time process, 169 
of lognormal distribution, 167 
of Poisson process, 252, 259 
properties, 165 


Wiener process, 226, 302 


zero-motion line, 85, 87, 128 
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Appendix A 


Introduction to solutions to exercises 


This solutions manual presents solutions to selected exercises. There are not solutions to 
all exercises simply as not all of the existing solutions are nicely typed, edited and checked 
for errors at this point. There are solutions to the following exercises. 


Chapter 1 — no exercises 

Chapter 2 — Exercises 2, 8 and 10 
Chapter 3 — Ex. 2, 5 and 7 
Chapter 4 — 1, 2, 6, 8 and 9 
Chapter 5 — 1, 4, 5 and 8 
Chapter 6 — 1 

Chapters 7 and 8 — 6 

Chapter 9 — 2, 7, 8 and 10 
Chapter 10 — 3, 4, 5, 12 and 13 
Chapter 11 — 1, 2 and 3 


Any book goes through various stages before actually being “born” in its printed 
version. This is also true for this book. It started with simple notes, went through its 
adolescence as various informal versions for the internet and became grown-up via the 
incorporation of serious and thoughtful comments by referees. 

I should probably mention that the solutions manual is not as well-tested as the text 
itself. While I believe that the solutions are correct, the ideas behind them are not 
explained in an as elaborated way as in the main text — after all, it is a manual. I 
am nevertheless sure that it will be very useful for users of this book. As always, any 
comments are welcome. 

The main text has been written word for word by the author of the book. The 
solutions of the exercises were all suggested by the author. A large group of collaborators 
have contributed, edited and refined them. They include Ken Sennewald, Christopher 
Kops and Michael Lamprecht. I am highly grateful to them for their support. 
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Appendix B 


Solution to exercises of chapter 2 


B.1 Solving by substitution — ex. 2 


From the main text (see equations (2.2.1), (2.1.2) and (2.1.3)) we already know the fol- 
lowing equations 


objective function: U; = yln a + (1 — y) In + (B.1.1) 
budget constraint in t: Ce = Wt — S (B.1.2) 
budget constraint in t + 1: Cee = Wey + (1 + re41) St. (B.1.3) 


Now the principle is to transform this optimization problem with constraints into one with- 
out constraints. We therefore insert the budget constraints (B.1.2) and (B.1.3) into our 
objective function (B.1.1). The unconstrained maximization problem then reads max U; 
by choosing s+, where 


U; = y ln(w; — s+) + (1 — y) In(ega + (1 + 141) 82)- 


The derivative with respect to savings s; is given by 


= (1 Tria) i ; 
Wt — St Wer + (1+ re41) Se 
which represents the first-order condition. When this is solved for savings s+, we obtain 
w 1— w 1- 1 
tl GiB) T Jag, SE Et Dag N 
LFT 14+ rest y y 
Wt+1 
= (1 — y)w: = =w Wi, 
t = ( ywi Tee t= Yt 


where W; is life-time income as defined after (2.2.3). 
To show that this is the same result as in (2.2.4) and (2.2.5) we compute first- and 
second-period consumption and find 


Ce = W — S = YW, 
Cot = Weer + (1 + reg) Ste = Weer + (1 + reer) (we — YW) = (1 + re) (1 oW, 


where the last equation used again the definition (2.2.3) of life-time income Wz. 
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B.2 General equilibrium — ex. 8 
Exercise 8 a) 
e The firms 


The technology is given by 
Y= ken] (B.2.1) 
As in section 2.4, we choose Y; as our numéraire good and normalize its price to unity, 
pı = 1. With this normalization, profits are given by m; = Y, — wk K — w}L. Sticking to 
section 2.4, we let firms act under perfect competition and get the following first-order 
conditions, 


A eee 

wë = aK, = aK; 151 5 (B.2.2) 
OY, (B.2. 

w= r 29 (1 _ a) K2L~*. (B.2.3) 


e Households 


The utility function is given by U; = Inc/ + 6Inc?,,. It is maximized subject to the 
intertemporal budget constraint 


= pY =1 0 
we = ce + (L Era) Cr 


Note that households work in the first period, but retire in the second. Hence, there 
is labour income only in the first period on the left-hand side and savings of the first 
period are used to finance consumption in the second period. Given that the present 
value of lifetime wage income is w+, we can compute individual consumption expenditure 
and savings, 


gies ts 
t= Tp t; 


Sı = W — O = i . gu (B.2.4) 


$ B 
Ci = Tp. + ripi) Ws, 


Assuming that all individuals within a generation are identical, then aggregate consump- 
tion in t is given by 


p 
Eg 


Ci = Lë + Le = ( (1 } rus) L. 


wW. 
1+8“ 
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e Goods market equilibrium 


Sticking to subsection 2.4.3, the goods market equilibrium requires that supply equals 
demand, Y; = C; + I, where demand is given by consumption plus gross investment. 
The next period’s capital stock is then given by Kı = J; + (1 — ô) Kı, where ô is the 
depreciation rate. These two equations imply 


Y; + (1 a ô) Ky = Ci + Kizi, (B.2.5) 
where we simply replaced gross investment by the goods market equilibrium. 
e Reduced form 


Providing that the technology used by firms to produce Y; has constant returns to 
scale, we obtain (see equations (2.4.3) and (2.4.4)) from Euler’s theorem that 


OY; OY; 
Y, = — Ki + — L. 
t OK, tT aL t 


and this yields with equations (B.2.2) and (B.2.3) the following relation Y, = w¥ K +wŁ L. 
Using this for (B.2.5) and splitting aggregate consumption into consumption of the young 
and consumption of the old gives 


wk K,+whLh + (1-6) K; = C? + Co + Kiss. 


Defining the interest rate r; as the difference between factor rewards for capital w and 
the depreciation rate 6, r; = wk — ô, we find 


rK + whL + K; = CY +C? + Kiar. 


As the old consume the capital stock plus interest C? = (1+ r;)Ay, we obtain 


B.2.4 
Kiy = WFL — Cf = (wf — dd) L a agit 


-i Ê a-a eke (B.2.6) 


S Kai = —— 
t+1 1+8 
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Exercise 8 b) 


The phase diagram depicts K+ on the horizontal and K¿+ı on the vertical axis. 
K 


ttl A 


t+1 


0 Ky K* K 
Exercise 8 c) 


e Steady state capital stock 


In the steady state, the capital stock is constant, K; = Kı = K*. Inserting in (B.2.6) 
yields 


K* = ail =a)" (K*" 


S (K = — (1 -a)l & K* = | 


e Steady state consumption level 
Substituting Kı = K; = K* into (B.2.5) gives 


Yia ok egik eerk. 


B.3 Difference equations — ex. 10 
Consider the following linear difference equation, 


Yai = aye +b, a<O0<b. 
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Exercise 10 a) 


Abe, 
1-a? 


A fixpoint y* satisfies y* = ay* + b, hence it is given by y* = where clearly a Æ 0. 


Exercise 10 b) 
Starting with yo, we obtain via solving by substitution 


Yı = ayo + b, 
Yo = ayı +b = a”yo + ab + b, 
y3 = ayo + b = ayo + (a° +a + 1)b, 


t—1 


y =a'yot b> a. 


i=0 


Consequently, the limit for t — oo is computed via 


: =. è t £ t—1 i 3 t 
Hm y E yo jim a j o ia D E yo jim a us 1-a’ 
where we had to assume |a| < 1 in order to use the limit of the geometric series. Never- 
theless, we have to distinguish between the following three cases: 


1. For —1 < a < 0 we have limo Yt = 0 + (eo EL 


l-a 
2. For a = —1, the series does not converge. 
3. For a < —1 the sequence is unbounded and alternating. If yo(1— a) > b, the 
subsequence {y2} diverges to co, whereas the subsequence {y2%+1 } diverges to —oo. 


If yo(1 —a) < b the subsequence {yo; } diverges to —oo and the subsequence { Yy2x41 } 
diverges to oo. 


Hence the fixpoint y* is only stable for —1 < a < 0. 
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Exercise 10 c) 
We need to distinguish between three cases here as well. 


(i) for —1 < a < 0 (left panel) and (ii) for a = —1 (right panel) 


Vist, 


Viet, 
b | ada b RF ve Ym 
4 
1 > > 
Yo om 7 yr Yo TA =r Yı 
Ya = ay, +b 
Yı = AY, +b 
v y 
(iii) for a < —1 
Yaa 
bor Jem Yri 
| I r 
b ib y, 
Yo l-a a 
V4, = ay, +b 


Appendix C 


Solution to exercises of chapter 3 


C.1 The envelope theorem II — ex. 2 


Exercise 2 a) 


The Bellman equation reads V (a;) = max,,{u(c;) + BV (az41)} and the derivative with 
respect to a; of the maximized Bellman equation reads 


daty 
dar í 


Using the budget constraint ayı = (1 + r) (at + wi — c+), we obtain 


dc; j dc; 
V"(a) = WO) a, + BV" (aes) +r) — a 


As we did not employ the envelope theorem, we can now use the first order condition 


in order to remove the derivatives of the control variable with respect to the state variable. 
This yields 


dc 
V' (as) = BV" (as1)(1 + ") Ta, + BV" (am) (1 +r) — T 


Oat 
= BV" (ar )(1 + ri) = BV" (ar) ae 


Exercise 2 b) 


Starting out with the Bellman equation V (M) = max,,{—e; + BV (Mi41)}, the deriv- 
ative with respect to M; of the maximized Bellman equation reads 


= Oey 
OM, 


OMi+1 
OM, ` 


V"(M:) = + PV’ (Mimi) 
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Not having used the envelope theorem, we see the derivatives wet 


in this equation. Using the constraint Mii; = f (e+) + Mi, we have 
Oe; 
OM, 


of the control variable 


V'(M) = + BV"(Mts1) (re H + i) ` 


But, as the first order condition is given by 1 = 8V'(M111) f’(ez), the expression above 
simplifies to V'(M;) = 6V'(Mi.1) which, due to its intertemporal nature, implies 


V"(Mis1) = BV’ (Meo). 


C.2 The standard saving problem — ex. 5 


Exercise 5 a) 


We start from the following two fundamentals of our problem. 


Utility function: U, = EL, Bulc) 
Budget constraint: kii = (tri + wt — c 


rE 
ome 
lis N 
b m 
Niit 


e DP 1: Bellman equation and first-order condition 


The utility function (C.2.1) can be rewritten as U; = u(c;) + BUi+ı and by applying 
Bellman’s idea we obtain 


V(k,) = max U; = max{u(c;) + BV (ke+1)$ (C.2.3) 
subject to (C.2.2). The first-order condition of maximizing with respect to c} reads 
d d dk 
ga ! P n = u (ct) + BV (kiy) =0 


as k1 = k1 (c) by the budget constraint (C.2.2). Since dki+1/dc = —1, we find as first 
order condition 


u (ci) = BV" (kipi). (C.2.4) 
e DP 2: Evolution of the costate variable 


Using the envelope theorem, the derivative of Bellman’s equation (C.2.3) reads 


dky41 (C.2.2) 
dk, 


V' (ki) = BV" (kii) PL+ r) V” (kiri). (C.2.5) 


e DP 3: Inserting first-order conditions 


Substituting the first order condition for t and t + 1 into equation (C.2.5) yields 
Bul (C41) = BO +r,) 6 tu (c+). Lagging this equation by one period finally gives u’ (c) = 
BL + rep1)u’ (Cipa). 
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Exercise 5 b) 


Now we use the Lagrange approach with an infinite sequence of constraints. Hence we 
have 
a, me {Bu (Cr) + Ar [k-41 oa (1+r-)k-— wy +c,]}. 


In order to make the procedure clearer, we rewrite the Lagrangian as 
L= ER, p tule) + DIA, [kr — (1+ 97)k, — wf + c] 

+ Age [ks — (1+ rs-1)ks-1 — w1 + Cs—1] 

+ às [ks+1 — (1 + rs)ks — wy + cs] 

+ 0% Ar [kry — (1+ r-)kr = wh +e]. 


This Lagrangian clearly shows the infinite number of constraints and multipliers. It also 
allows us to write the first-order conditions for time s as 


Le, = BP ‘ul (es) + As =0 4 às = —u' (cs) 8, 


\5— 
~=14+7,, 


Xs 


jem” Cea Ges Arte 
Loa 


Combining the first and second first-order conditions gives 


Hence, by lagging this result one period, we obtain 
u (cs) = BL + rs41)u'(Cs41). 
Exercise 5 c) 


We can also first derive an intertemporal budget constraint. To this end, we rewrite 


ites, : ; 
the budget constraint (C.2.2) as ky = Kester“ ut on ag kipi = Feriti t tti, Substituting 
y : ltr: 1+rt+i 
sufficiently often yields 


kt+2+ct41 -wi 


— wt L L 
pE Tr ATS We ki42 + Cipi — Wha , a We 
t | 
1+7 (1 + re) + rt) 1+r 
kt+3+ct42— Wio L 
pn L 
= 1+rt+2 Ct+1 — W441 i Ct — UW; 
j 
(1+ rei) +r) l+r 
L L L 
ktt + Ceo — Wes Ci We Ce Uy 


(+ rea) trer) (+r) ltr 


SiS iim 
i— oo (1 + Tt414-1) Bi ies (1 + ra) (l +r) 


kipi 3 Ceti — Wii 
(1+ reri) Frell H r) 
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and thus we obtain 


mott Ss 
RRS fad + Tits) i=0 IÉ- + Tits) 


kr= 


Using the no-Ponzi game condition (see ch. 3.5.2), we assume that the limit is zero. Then, 
by substituting T = t + i, we obtain 


L L 
ki — ynoo Cr — W7 — ynoo Cr oo Ww, 


a es Tt = Daian Tt = ey Tt 
L1 + Tits) L2 + fits) Lo + Tits) 


and the desired intertemporal budget constraint with a constant interest rate reads 


Se (Ltr) Ne, = ke + US (1 +r) OE, 


T 


Hence, the Lagrangian reads 


L = ER pule) +A [ELA Hr OMe, = ky EL ry Dw] 


T 


where A is the Lagrange multiplier. 
First-order conditions are 


È: = Btu (c) a Tal als rA = 0, 
Ly =0. 


The first-order condition for period T = t and for period T = t + 1 read 


u (c) +A +r) = 0, 
Bu (e) +A +r)? =0, 


and dividing them gives 


u (c) 
Bul (Ce41) 


=(lt+r) => wa) = (1+ ru (c). 


C.3 A benevolent central planner — ex. 7 
Exercise 7 a) 
Our starting point consists of the following equations, 


Social welfare function: Var ate) 


Resource constraint: Kai — kh =Y (Ki, Li) — ôK; — C. 


a0 
bo čo 
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e DP 1: Bellman equation and first-order condition 


The value function for this problem reads V (K,) = maxyc,} U; s-t. (C.3.2) and the 
Bellman equation reads V (Ky) = maxo, lu (Ct) + BV (Ki+1)|. The first-order condition 
of maximizing with respect to C; reads 


dKiyı 
dC; 


u (Cy) + BV" (Kisa) =0 


as Kı = Kr4i1(C;) by the resource constraint (C.3.2). From the budget constraint we 
see that dky41/dC; = —1, and therefore the first-order condition is 
u (Cà = BV (Kii). (C.3.3) 
e DP 2: Evolution of the costate variable 


Since C; = C;(K;41) by first-order condition (C.3.3) and resource constraint (C.3.2), 
we get V (Kiy1) = u (Ci (Kir1)) + BV (Ki42). Employing the envelope theorem, we have 
dKt+9 
dK is. 


V’ (Kuyt) = BV" (Kiza) 


Employing the resource constraint Kio = Y (Kipi, Ley) + Kii — K1 — Ca, this is 
equal to 


ð 
V’ (Ki) = BV! (Kiso) (= ae 5) ; (C.3.4) 


e DP 3: Inserting first-order conditions 


From (C.3.3) we know that wu! (C1) = BV’ (Ki+2). Hence, equation (C.3.4) reads 
V' (Kii) = wu! (Cry) Can +1- 5). Using this in (C.3.3) again, we find u’(C;) = 


Bu! (C441) (2 +1— 5) and thus 


ul (Ci) 1 
= ; C.3.5 
Bu! (C441) ee ( ) 
ƏKt41 


Exercise 7 b) 


The main difference between exercise 5 and 7 of chapter 3 is that the first one considers 
the optimal saving problem of a household in a decentralized economy. This exercise 
considers optimal saving of a central planner. 

When we compare the Euler equations of the two setups, we find that they are the 
same if =>— — ô = rą, ie. if the marginal productivity of capital corrected for the 
depreciation rate equals the interest rate individuals face. One condition for the factor 
allocation to be identical in the planner solution and in the decentralized solution is 
therefore that the interest rate in the decentralized solution obeys this equation. 
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Exercise 7 c) 


Two standard versions of instantaneous utility functions are the following, 


Logarithmic utility function: u(C,) = ln C; (C.3.6) 
Ci? —1 
CES utility function: wh.) = = (C.3.7) 


If we want to substitute the logarithmic utility function (C.3.6) into (C.3.5), we first 
compute marginal utility, which gives u’(C,) = ¢C7t. We then obtain 


p 
I 


OY 
aKa? 


Cipi = Ci 


Let us now use the CES utility function (C.3.7) instead for (C.3.5). Computing mar- 
ginal utility gives u/(C,) = C77 and we obtain a linear difference equation in consumption 
which reads 


Cry = p Ci. 


oY 
aKa? 


Appendix D 


Solution to exercises of chapter 4 


D.1 Phase diagram I — ex. 1 


Exercise 1 a) 
We first construct zero-motion lines and find 
tı =0 S f (x1, £2) =0, t2=0 & g(z1, £2) =0 


The slope of the zero-motion lines follows from the properties of f (.) and g (.) as stated 


in the problem, 
dx» dx2 


a <0, — > 0. 
E01 f(2y,-29)=0 d1 | g(x4,20)=0 
The zero-motion lines can be linear or non-linear. The following figure draws the linear 
case. 
X, 


- 


X=2(X),X.)=0 


aie 


x,=f(x,,x,)=0 


Xı 
Figure D.1.1 Phase diagram I 
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Since fr, < 0, gr. < 0 are given and the implicit function theorem (see equation 
(2.3.3)) states 


dx, Oh(.) /Oxr2 


we have fr, < 0 and gz, > 0. Hence, pairs of arrows are explained by 


fe, < 0 : Starting from the tı = f (x1, £2) = 0 line, a small movement to the right 
will decrease the value of tı. Hence, for any points to the right of f (x1, £2) = 0, xy 
is decreasing, i.e. tı < 0. 


fr, < 0: Starting from the tı = f (x1, 22) = 0 line, a small movement downwards 
will increase the value of tı. Hence, for any points below f(21,72) = 0, zı is 
increasing, i.e. tı > 0. 


Jey < 0: Starting from the #2 = g (x1, £2) = 0 line, a small movement upwards will 
decrease the value of #2. Hence, for any points above g(x1, £2) = 0, z2 is decreasing, 
i.e. L < 0. 


Je, > 0: Starting from the t2 = g (£1, £2) = 0 line, a small movement to the right 
will increase the value of «2. Hence, for any points to the right of g(x1, £2) = 0, £2 
is increasing, i.e. t > 0. 


Exercise 1 b) 


A phase diagram analysis allows to identify a saddle point. If no saddle point can be 
identified, it is generally not possible to distinguish between a node, focus or center. If 
the given differential equation system was linear, we could use the approach in section 4.5 
via eigenvalues. 


D.2 Phase diagram II — ex. 2 


Exercise 2 a) 


Zero motion lines are given by t = 0 & y = afz and y = 0 & y = b. The partial 
derivatives indicating the slope are oe =y>0, 2 a, =¥>9, 2i = ] > 0 and — zi =-l< 
0. Depending on the sign of b, we get the following two phase diagrams. 
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x y 


Figure D.2.1 Phase diagram IT 
Exercise 2 b) 


Generally speaking, a phase diagram analysis allows to identify a saddle point. If no 
saddle point can be identified, it is generally not possible to distinguish between a node, 
focus or center. If the given differential equation system was linear, we could use the 
approach in section (4.5) via eigenvalues. 

The left panel shows the case of a saddle path. The right panel does not display a 
fixpoint. 


Exercise 2 c) 
Let us consider the differential equation system 


t(T) = z(T)y(T) = a 
u(r) =y(T) -b 


with T > to, to > 0. The initial value of the system is (x (to) ,y (to)) = (Xo, yo). The 
second element of our system, describes the evolution of the function y in time. Because 
y evolves independently of x and the differential equation y(7) = y(T) — b is linear, we 
can use the backward solution (4.3.7) to get the solution 


(D.2.1) 


g(t) = yor” © + ofl — ee *] = (yo — b)et™ +b (D.2.2) 


with t € [to, 00). 
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Now we can use the solution (D.2.2) for the first element of the system (D.2.1) and 
get 
&(r) = U(r)x(T) — a = ((yo — b)e™ +b) a(7) — a (D.2.3) 


with T > to. Therefore the solution & : fto, o0) — R is described by a linear differential 
equation (D.2.3). We can use again the backward solution (4.3.7) to get the function Z. 
Before we apply the backward solution, we determine the integral 


Art) = | y(u)du = f [(yo — bye“ + b] du = (yo — b) [e — e™] + b(t — 7) 


with 7,t € [to,co) and r < t. If we insert this result in the backward solution of the 
differential equation (D.2.3), we obtain the solution 


t 
#(t) = roe tt — af EAD dr (D.2.4) 
to 


or more explicitly 
E(t) = axoexp {(yo — b) (e” © — 1) + B(t — to)} 


-a f exp { (yo — b) (e7 — 1) + b(t — 7) } dr. 


to 


Summarizing, we have found the solution (D.2.4) and (D.2.2) of the differential equation 
system (D.2.1) with initial value (£o, yo). 
D.3 Solving linear differential equations — ex. 6 


We start from the differential equation ay(t) + By(t) = y, with the boundary condition 
y(s) = 17. Assuming a ¥ 0 and dividing by a yields 


, b EN 
y(t) = —~ y(t) + Z S a) = ay(t) +b, (D.3.1) 
where we defined a = —G/a and b = y/a. (For the case of a = 0, we have a trivial 


problem.) Since a, 3,7 are constants, a and b are constants as well. Now from the main 
text (see equation (4.3.6)) we know that the general solution of the differential equation 


(D.3.1) reads 
y (t) = ef Ket € + l eT S Odih (t) it) 


where y is some arbitrary constant. 

In order to obtain one particular solution, some value y(s) at some point in time s 
has to be fixed. Depending on whether s lies in the future t < s or in the past s < t, the 
equation is solved forward or backward. 
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Exercise 6 a) 


Since t > s we use the backward solution. Our initial condition is y(s) = 17. Then 
the solution of (D.3.1) is 


t 
y(t) = 1el eer 4 f elt Ddug (r)dr, (D.3.2) 


according to equation (4.3.7) from the main text. But since a,b are constants, we have 


t 
b 
y(t) = {je bf ef )4gr = 17e-9)4 — — [ele] 
s a 
= Terr = 4 [1 — e6799] = e417 + 2 =e 
a a a 
_ (s-t)2 y y 
= e“ a = ee, 
erg ig 
where we used a = —G/a and b = y/a. 


Exercise 6 b) 


Since t < s we use the forward solution. Our terminal condition is y(s) = 17. Then 
the solution of (D.3.1) is 


y(t) = 17e h emer — l e- aduh (7) dr, (D.3.3) 


t 


according to equation (4.3.8) from the main text. But since a,b are constants, we have 


s 
t 


2 b 
yty= Pe Oe bf e (Had, = 17e-#)4 4 [ee 
a 


t 


b b b 
= 17¢e(t-s)4 a eal [ef "2 = 1] = Lag ats =) us 
a a a 
i Caer u a 
=e€ a -+ ; 
e æ 
where we replaced again according to a = —8/a and b = y/a. 


Exercise 6 c) 


Obviously we used the forward solution in case t < s and the backward solution in 
case t > s. However, both solutions coincide as anticipated, since swapping the integral 
limits went along with changing the algebraic sign of the corresponding integral (compare 
(D.3.2) to (D.3.3)). 
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D.4 Derivatives of integrals — ex. 8 


Exercise 8 a) 
Using Leibniz’s rule from equation (4.3.3) yields 


=| f toas] = a fy) -aO f aio = fly) 


Exercise 8 b) 


Here we use Leibniz’s rule again and obtain 


d Y _ oy _ da vd 
5 | S" roas] = Z tow Rios ESTE 


Nae 
=1 =0 
Exercise 8 c) 


Note that there is a trick in this exercise. Write the integral as T f (z) dz, i.e. just 
use a variable different from y as variable of integration. Then, using Leibniz’s rule, we 


have ; f r i a 
a 
tlf roel = O- Fiat | Ef edz=o. 


=0 =0 =0 


Another way of putting this is iy Gigi (y) dy| = iy |F (b) — F (a)|, where F (.) is the 


indefinite integral of f (.). Then, as F (b) — F (a) is a constant, the derivative with respect 
to y is obviously zero. 


Exercise 8 d) 


From equation (4.3.2) in the main text we know the definition of an indefinite integral 
and thus have 


Erw =E [twd=F), 


where F'(y) is called antiderivative, primitive or indefinite integral of f(y). 
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Exercise 8 e) 


The following proof is similar to the one for equation (4.3.4) in the main text. Consider 
the product rule 


(x(t)y(t))' = a'(t)y(t) + v(t)y'(t). 
Integrating both sides yields 


f cowumyae=famuoaes f Oroa 


as the integral of a linear combination is the linear combination of the integrals. It follows 


that 
J Dal = | rawea f x(t)y'(t)dt 
e | OOd = BOOR- f OOd 


D.5 Intertemporal and dynamic budget constraints 
— ex. 9 


Exercise 9 a) 


We start from the following two budget constraints (BCs), 


intertemporal BC: E(t) + A(t) = r(t) A(t) + I(t), (D.5.1) 
dynamic BC: l D,(T)E(T)dr = A(t) + i) D,(7)L(r)dr, (D.5.2) 
t t 
where D, defined as 
D,(T) = exp - f r(s)as| ! (D.5.3) 
t 
We have to show that (D.5.1) implies (D.5.2), if and only if 
T 
jm A(T) exp -f (r)a =0. (D.5.4) 


We start by rewriting (D.5.1) as A(t) = r (t) A(t) + I (t) — E(t) and thus recognize 
that it is a linear differential equation in assets A(t). Its general forward solution by 
(4.3.8) is given by 


A(t) = A(T)e~ JE r@ar [ (I(t) — Blr)}e- POLTI 
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Inserting D,(7) via expression (D.5.3) yields 


P T 
e D,(T)I(T)dr + A(t) = A(T)e7 h "Oe +f D,(7)E (7) dr 
t 


since the integral of a linear combination is the linear combination of the integrals. Letting 
T go to infinity we obtain 


r D,(T)I(T)dr + A(t) = P A(T )e7 e r(r)ar +f D,( 


and this equation coincides with the dynamic budget constraint (D.5.2) if and only if 
equation (D.5.4) is valid. 


Exercise 9 b) 


By defining 
f(r) = D,(T)E(T), g(r) = Dr(r)L(7) (D.5.5) 


the intertemporal budget constraint (D.5.1) simplifies to 


a f(r)dr = A(t) + i g(t)dr. (D.5.6) 


Differentiating equation (D.5.1) with respect to t yields 


af far gati f ar 


and using Leibniz’s rule from (4.3.3) we have 


-40+ f Aroa = Aw - g(t) f Sarar 


Now, by employing (D.5.5), we obtain 


—D,(t)E(t) + i. £ (D,.(r)E(r)) dr = A(t) — D,(t)I(t) + i. T (D, (T)I(T)) dr 


and since E(T) and I(T) do not depend on t, we have 
—D,(t)E(t) + f E(@)ÉD,(r)dr = A(t) — D,(t)I(t) + i WOL O 
t t 


Considering (D.5.3) we find 


Sos fol fee] =e f roa EEn 


= r(jexp |- [r(s)as] =r0D0) 
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where we used Leibniz’s rule (4.3.3) again. Furthermore we notice that 
t 
D,(t) = exp -f r(s)ds| = exp(0) = 1. 
t 
We therefore have 
—E(t)+ ro f E(r)D,(r)dr = A(t) — I(t) + r(t) I(r) D,(7)dr. 
t 


Using (D.5.1) for the integrals, we finally find —E(t) + r(t)A(t) = A(t) — I(t). This 
obviously coincides with the dynamic budget constraint (D.5.2). 


344 Appendix D. Solution to exercises of chapter 4 


Appendix E 


Solution to exercises of chapter 5 


E.1 Optimal consumption over an infinite horizon — 
ex. 1 


Exercise 1 a) 
The present value Hamiltonian reads 
HP =e PI ne(r) + (7) [r (7) A(7) + w (7) — (rr) e(7)]. 
Hence, optimality conditions are 


e Pitt] 


HP=06 = np (E.1.1) 
c 
i= -H} OH = nr. (E.1.2) 
Applying the logarithm to (E.1.1) yields — pir- t] — Inc = nn + ln p. Computing the 
derivative with respect to time T gives —p — ¿ = i + È as variable are functions of time 
T, ie. c= c(T), n =n(T) and p = p(T). With (E.1.2), 
Ċ p 
-=r = p. 
c P 


We are now going to derive the same result with the current value Hamiltonian. This 
one reads H = Inc(r) + A(T) [r(T)A(T) + w(T) — p(T)e(T)|. Optimality conditions are 


1 
H. =0 & -= àp (E.1.3) 
c 
À= på- Ha À= pr\— Pd. (E.1.4) 
Applying the a to (E.1.3) and then computing the derivative with respect to time 
T gives —£ — ; = +. Hence with (E.1.4) we have 
cC P c P 
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Exercise 1 b) 
The present value Hamiltonian reads 
H? Se EESE A CEUS peT) 
Hence optimality conditions are 


H? =0 ee = E.1.5 
=0s = A 
c u! (c) np ( ) 


ġ=-H} oOo = nr. (E.1.6) 


Applying the logarithm to (E.1.5) yields —p [r — t] — In u'(c) = nn +Inp. Computing the 
derivative with respect to time T gives —p — vè = 7+ 5 as u(c) = u(e(T)), n = (7) and 
p = p(T). With (E.1.6), 


u” 


ee ee 
et oe 
Now again with the current value Hamiltonian, H = u (c) + A|rA+w — pe]. Opti- 
mality conditions are 
H. = 0 Su (c) = àp (E.1.7) 


À= pà- Ha À= prA— Ar. (E.1.8) 


Applying the logarithm to (E.1.7) and then computing the derivative yields we = A + r, 
With (E.1.8) we therefore obtain 


E.2 Optimal consumption — ex. 4 


Exercise 4 a) 


From the main text in (5.6.4) and exercise 1 b of ch. 5, we know the so called Keynes- 
Ramsey rule (E.1.9). With the given CES utility function, 


l-o _ 1 
u(e(r)) = OZ, (B21) 
we get u’ (T) =c(r)~? and u” (T) = —oc(T) ®t. The optimality condition then reads 
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Exercise 4 b) 


Let us first recapitulate L’H6pital’s Rule. Let f(x) and g(x) be two real functions 
twice continuously differentiable. Suppose that the limits of both functions as x ap- 
proaches x* are 0, i.e. lim, ..« |f (x)| = lim,..« [g (x)| = 0. Then L’H6pital’s rule implies 


lim Ga = lim (4 a). 
e g(a) ) ene gi) 
This rule applies to the indeterminate forms 0/0 and oo/oo. 


Now define f (o) = c!-? — 1 and g(o) = 1 — o and apply L’Hopital’s Rule for o* = 1 
to the given CES utility function. We get 


_ ¢ %—1 pHopital,. c' 7 (—Inc) 
DEO ae ay 


where we have used 


l-o 


L = oe = e07) a (1—o)Inc=c'? (—Ince). 


Exercise 4 c) 


Computing the first two derivatives of the CES utility function (E.2.1) yields w’ (c(7)) = 
c(t) > 0 and u” (e(T)) = —oc(r)~? | < 0. The first derivative is positive for any values 
of o (and positive c(7)). The second derivative is negative for all positive ø. Hence, o > 1 
still implies decreasing marginal utility from consumption even though instantaneous util- 
ity (E.2.1) is negative for ø > 1. 


Exercise 4 d) 


Ue(t)/Ue(t¢a) d(c(t+A)/c(t)) 
ECRAN aleea) 
We compute the derivatives as in (2.2.11) using the instantaneous utility function u (c (T)) = 


e(r)! 77 — 
or) from (E.2.1) as 


We express the definition in (2.2.9) for t and t+ A as €44 = 


otic SOI EO ea) d(c(t + A) /e(é)) . 
ee c(t + A) /c(t) d(y[1—a]e(t)’/(-y(1-a)e(t+A)”) 


Following the same steps as after (2.2.11) in discrete time, we find e444 = 1/0. 
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E.3 A central planner — ex. 5 


Exercise 5 a) 


We start from the following setup, 


i oo Cl-¢ =f 
Social welfare function: U= i Ot ea eo, (E.3.1) 
t —o 


EU resource constraint: K=BK-C. (E.3.2) 


The Hamiltonian then reads H = (C!~* — 1) (1 —0) ‘+A (BK — C) and the correspond- 
ing optimality conditions are 


CY’ =A -olh = lnnà > -$ = (E.3.3) 


A 


i= -ABe2=p-B. 


Combining them yields the consumption growth rate C/C = o™! (B — p). 
Exercise 5 b) 


Consider the linear differential equation t (7) = a (T) x (T) + b(7) and its backward 
solution 


z (T) = reft 94s + | els aWdup (s) ds. 
t 
Given the growth rate of consumption, 


Xo B= ; BS 
a= Faya E 


Oo Oo 


C (T), (E.3.4) 


we can define b(T) = 0, a(r) = B-p and x(t) = C(r).The backward solution to the 
differential equation (E.3.4) is then obviously 


C (T) = Cea Cn (E.3.5) 


E.3. A central planner — ex. 5 349 


Using this for the utility function, we express the latter as 


i. elt (crete) 1—0 7 1 1 ag 
t 1—0 


U (t) 


l-o 
l-—o love) love) 
Gs / ela- egr — 1 [e?e] dr 
1 Sg: t 1 = g t 
l-o lee) 
_& / [20-01 1 [ee] 
l-o J, 1 — ø —p t 
1—0 love) 
mS aa ace Te) 
l-o J; (l—o)p 


We can now conclude that the utility function is bounded if 
p 


B-p 
l-o)-p<0SB< —. E.3.6 
a Ueaap == (E.3.6) 


As an illustration, consider these two exponential functions: 
e, er. 
| 1 i oe 


> > 
x x 


The integral from, say, 0 to infinitive of the curve e” in the left panel is infinite as the 
exponential function is ever increasing and the area below the graph is therefore ever 
growing the further one goes “to the right”, i.e. the larger x. The area of the curve e`” in 
the right panel “can be hoped” to be finite as the distance of the graph to the horizontal 
axis is becoming smaller and smaller. The area below the graph is also increasing the 
larger the right boundary is, but it is, as one can easily show analytically, bounded from 
above. 

A second condition, now a more economic one, one can impose on the problem is a 
positive growth rate, see (E.3.4). This is not necessary, however, but makes sense from 
an empirical perspective. 

o '(B-p)>0SB>p. (E.3.7) 


These two constraints can be plotted into the following figure. For illustration purposes 
we assume that @ < 1. Otherwise the result would have to be adjusted following the 
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condition for B from (E.3.6). In the shaded area, there is growth while the utility function 
is still bounded. 


p= 
B l-o 


45° 


Figure E.3.1 Boundedness condition 
Exercise 5 c) 
The solution proceeds in three steps 
e ist step: Solve for the general solution of K (7) 
The goods market clearing condition reads 
K (1) = BK (T) - C (1) “SB” K = BK — Cie, 


where 
g=oa'(B—p). (E.3.8) 


The backward solution is 


K (T) = eP K, = A ePIC esl- ds = eP- kK, = C, a esl9—B]-gt+ Brg 
t t 


Now, since 
| " (slo-Bl-at-+Br [ere a ee 
t eee: g-B 
we obtain za ce 
T—t)g _ p(T— 
K (T) = eo K, = Ce 
—B 
and thus P eS 
T—t)g 
K (T) = e0798 [K = z| = > a (E.3.9) 
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e 2nd step: Use the No-Ponzi-Game condition 
The No-Ponzi-Game condition for the budget constraint 


lim e-°"\(r) K (r) =0 “SS” lim eC (r) K (r) =0 


(E25) Jim eP" h ea = K(r)=0% lim Cre (8-0-9) K (7) =0 


e 3rd step: Insert (E.3.9) into No-Ponzi-Game condition 


Tt 
lim Of 7PM rae LK, -S | a =| =0 
g 


T—00 -B g- B 
oo COT xe -2CR rete pei 
= lim C; { [Ae 52, | e -p p—g)t-(B-g) bo (E.3.10) 


Since lim,_,.,e~” = 0 and lim,_.., €” = oo, the second term in the curly brackets only 


vanishes if B — g > 0. (Note that (B — p — g)t is constant as we consider a fixed t and 
the limit for T.) The difference between B and g is positive with (E.3.8) iff B — (=£) > 
0 B < ;®&. This is our condition (E.3.6). 


The first term in the curly brackets only vanishes if 
C: 


Kı = E.3.11 
pS (E.3.11) 


This implies with (E.3.9) 


C: 


pa (E.3.11) pe 
= Bg“ t)g = Kel tg, (E.3.12) 


K (T) 


The growth rate of capital equals the consumption growth rate at every moment in time, 
K (T) /K (T) =C (7) /C (7) = g. This is a standard result of the Rebelo AK model. 


Exercise 5 d) 


You could resign from your job without making anyone less happy than before if 
perfect competition prevailed and in the absence of any other distortions. Since with 
perfect competition the decentralized economy also reaches the optimal solution. There 
is, therefore, no need for a central planner. 
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E.4 The Ramsey growth model — ex. 8 


Exercise 8 a) 


We start by looking at the condition for optimal consumption growth. Given the 
nee 


utility function U (t) = f° el"! cat 


oO 


dr and the constraint 
K =Y (K,L)- C- ôK, (E.4.1) 


the Hamiltonian reads H = eo +A (T) [Y (K (7), £) — 6K (T) — C(7)]. Optimality 
conditions are 


Cc HX 
He =C" ~\=08 07 =)> -05 =$ 


: H 
opie” 


À 
ag = PAT AIYK — 8) y =P -Y¥K +8 


Combining these two conditions gives the consumption growth rate 


C Ye-6-p 


A - (E.4.2) 


We now consider the phase diagram analysis. Equations (E.4.1) and (E.4.2) represent 
a two-dimensional differential equation system which, given two boundary conditions, give 
a unique solution for time paths C (T) and K (T). These two equations can be analyzed 
in a phase diagram. 


Zero-motion lines are 


K (1) 
G(r) 


> 06C<Y(K,L)—5K, 
> OS YK >O+ 9, 


for the general case. An example for a Cobb-Douglas production function Y = K°L1~°% 
reads 


é I-a 
C (T) > 0S aKIL" > +p K*< =) L 
+p 
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Plotting zero motion lines and corresponding arrow-pairs yields the following figure. 


c) 


0 * K(t) 


We see that the fix-point is a saddle point. 
Exercise 8 b) 


_ Concerning boundedness of the utility function, the growth rate of consumption is 
a = -*, where r = Yg — 6. Hence C evolves according to 


Ce) = Cee 2 


if we are willing to assume a constant interest rate r. We can do this for the question at 
hand by setting r equal to its upper bound during the course of capital accumulation (i.e. 
its level at the initial capital stock Ko). 

The utility function can then be expressed as 


r—p l-o 
U(t) = [Pere |(c) = 1 


1 = r 
= if ciel 2(1-0)—p](r—t) ee Dhar 
Cr? oo |== (1-o)—p|(r-t COT —p(r—-t 
z Jog t el P| Jdr = Log t le pl )] dt 
S Mms 
=M 
1 1 oo 1 
ZG _-_ f,-e(t-t) =M = 
(9 eee a -e 
l—o 
ae 1 C; oo [32 (1-2) -e] (7-4) ge 
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We can now conclude that the utility function is bounded if 


r — 
a EE E E = 
o l-o 
As we assumed r to represent the highest interest rate for all capital stocks between Ko 
and the steady state, this is a sufficient condition for boundedness. It is not necessary 
and can be relaxed. 


Appendix F 


Solution to exercises of chapter 6 


F.1 The envelope theorem once again — ex. 1 


From the main text in (6.1.4) and (6.1.5), we already know that the maximized Bellman 
equation is 
pV (a(t) = u(c(a(t))) + V'(a(t))[ra(t) + w — pe(a(t))). 
Computing the derivative with respect to assets a(t) yields 
PV'(a(t)) = u'(ela(t)))e(a(t)) + V"(a(t)) [ra(t) + w — pe(a(t))] 
+ V"(a(t))[r — pe'(a(t))]. 
As no envelope theorem was used here, we see the derivative of the control variable with 


respect to the state variable. 
Using the first-order condition (6.1.5), which reads wu/(c(t)) = pV‘(a(t)), yields the 


derivative 
pV'(a(t)) = V"(a(t))[ra(t) + w — pe(a(t))] + V"(a(t))r. 
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Appendix G 


Solution to exercises of chapter 7 
and 8 


G.1 Closed-form solution for a CRRA utility func- 
tion — ex. 6 


Exercise 6 a) 


Let there be an utility function 
U; = Ey yee 7 + (1 — y) et] = ye? + (1 y) Eer. 


Substituting again savings from the budget constraints (8.1.6) and (8.1.7) gives an ex- 
pression similar to (8.1.10), U = y |w; — s}? + (1 — y) E [((1 + rest) ae 4 , where 
expectations need to be formed with respect to the term in parenthesis, ((1 + r141) s)? 
As only the interest rate is uncertain from the perspective of t but savings s, are not (as 
it is the variable under control of the household), we can express preferences by 


U = y [we — si]? +1 — 7) 7-76 


where ® = E; [(1 + Tei) | ; 
The first-order condition with respect to s+ is then 


(1-0) y[we— s]? = (1-0) [1-9] 57% 


EEEE -mlsya}’ 


e =y] pan 
where the last equality defined the intertemporal elasticity of substitution ¢ = L, 


Note the close similarity of the first-order condition here to the one in the logarithmic 
stochastic case in (8.1.11): The factor ® is new here and the parameter o which captures 
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the degree of intertemporal elasticity of substitution. Solving for savings gives 


st i- {5} ] = W645 = aes (G.1.1) 


With (8.1.7), consumption when old is then 


Wt 

& 
ss 
ace} 


which shows that consumption in t + 1 is uncertain indeed, as rı is uncertain. Con- 
sumption as young is from (8.1.6) 


Ce = [L + resi] se = (1 + rea) 
1+4 


Exercise 6 b) 


With £ = 1, implying © = 1 because of ø = 1, we recover the logarithmic solution in 
(8.1.12) as 
me Wt Ne Wt 

1 {ey} eer 

When individuals make saving decisions in t the interest rı they get for their savings 
in t+ 1 is uncertain as interest r;,,; and hence consumption c;,, are realized at the end 
of period t + 1. As people don’t know what interest will be like in t + 1 they make 
expectations in order to determine their savings in t. 

Consumption c is not uncertain from the perspective t as it is realized in t. If we are 
in t— 1, & is uncertain for the same reason why c+, in t is uncertain. 


St = [1 — y] w 


Appendix H 


Solutions to exercises of chapter 9 


H.1 A household maximization problem — ex. 2 


We consider a standard consumption-saving problem of a household, max,.,} ., Ut, where 


}r>t 
U; = BEB" (cr) (1.1) 
subject to kpy1 = (1 + r+) kr + Ot — fc. 


The household chooses his consumption path {c,} thus he maximizes his intertemporal 


utility U;. We can solve this problem (H.1.1) by using dynamic programming. This 
solution method can be split into three parts. 


e DP 1: Bellman equation and first-order condition 
The Bellman equation has the following form 
V (ki) = — {u (cz) + BE, (V (Ke41))$ (H.1.2) 


with V(.) the value function of our problem (H.1.1). Therefore the first-order condition 
is 


ul (ce) = BE: (V’ (ke41)) A (H.1.3) 
t 
e DP 2: Evolution of the costate variable 


The derivative of the Bellman equation with respect to the capital stock of the house- 
hold gives (using the envelope theorem) 


V"(ki) = BE: (V"(key1)) (1 + re) (H.1.4) 
e DP 3: Inserting of the first-order condition 


Finally, we insert the first-order condition (H.1.3) in the equation (H.1.4) and yield 
the Euler equation 
Bul (Ci) 
(L respi) ‘Pepa / Utta 


Pe/ Vt 
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H.2 Labour demand under adjustment cost — ex. 7 


The general objective function is given by I]; = nee Instantaneous profits 


amount to 
a, =F (L,) -—w,L, — ®(L, — L,-1) (H.2.1) 

where w,L, labor cost and ® (L, — L£,_1) adjustment cost in 7r. The normal assumption 
is that both hiring and firing individuals induces costs so a typical functional form is a 
quadratic form like ® (L, — L,_,) = g |L, — EAE 

The objective function reads from inserting (H.2.1) into the objective function (9.5.6) 
F(L,) —w,L, — Ẹ (L, — Lr) 

(1 a ry? 

Computing the derivative with respect to L, gives 


E [2 (L-)— wr — P (Lr — Lr) | $ (Lr — a 9 

T (1 +r) l (1 +r) À 
We have index 7 in the expectation because when we have to decide about employment in 
T we have all information available in 7. The derivative shows that choosing employment 
for r affects the firm in two periods: in period 7 where L, is chosen, where it determines 
output and where L, shows up in the adjustment cost function ® and in period 7 + 1, 
where it shows up again in the adjustment cost function when optimal employment for 
T + 1 is chosen. 

When is L, chosen? In r. Hence, we form expectations in 7 and all quantities with 

index 7 are known. The first-order condition can therefore be rewritten as 

F' (L,) — w, — ®' (L; — L,_1) | E,®' (L,41 — L,) 

(1 Ley (1 Fryti 

There is only uncertainty about L-+ı which is uncertain as the wage in T + 1 is unknown 
and Lı will be adjusted accordingly in 7+1. Hence, expectations apply only to the first 
adjustment cost term and we can simplify the first-order condition further to 


Il, = E D 


= 0. 


P (Lry — L- 
E, Codi dar (L) — w, @ (L, — L.) = 0. 
1l+r 
Expressed for t and after rearranging, we have the equation in the text, 
P (Lisi L 
FL ty Oo = hea ye 
When we specify ® (L; — Li) = $ (Lı — Ley , we obtain 
$ [Liri — Le] 


F' (Li) = wi + (Li — Lei] — E: Lop 


Additional production must pay additional wages and additional costs today but we sub- 
tract expected adjustment costs tomorrow. 

When there were no adjustment cost, i.e. ọ = 0, we have F” (L;) = w;. Employment 
is chosen such that marginal productivity equals the real wage. 
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H.3 Solving by substitution — ex. 8 
Utility is given by Up = EpX92)87u(c,) and individuals maximize it with respect to 


(1 a 1) a, — Ar41 + Wr 
Pr l 


ûr+1 = (1+r-) ar + Ww — Prey € Cr = 


Replacing c, in the utility function gives 


1 = 
Up = E252 bu (‘ +17) Ap — ar41 + =) 


Pr 


We now can take the derivative with respect to any a41, because by optimizing with 
respect to the state variable a,,, we implicitly choose c,. Therefore 


ar 1++r 
Eob u (c+) a= Eo tu (Cr41) — L 
Pr Pr+1 


H.4 Optimal training for a marathon — ex. 10 


The sport event will take place in some future point in time, say in m days, i.e. in t+ m 
where t is today. So 7 € [t,t + m]. There is a need for training e, as fitness increases with 
training in the following way 


Fri1 = (1—6,) F, +e, 


while fitness also decreases over time with depreciation rate 6,,0 < 6, < 1. As the 
individual dislikes training, utility is decreasing in training effort e,, i.e. u’ (e,) < 0. But 
the fitter the individual is, the higher happiness of an individual h (Fiim) in t+ m as it 
will be more successful then. 

With negative marginal utility of effort we can assume either increasing or decreasing 
marginal utility. Both might be reasonable. In the first case, additional training causes 
less disutility, the argument can be that we have to warm up and then the fourth kilometer 
might be easier than the first kilometer. But also the second case is realistic as the first 
kilometer might cause less disutility than the 20th kilometer. The assumption of increasing 
or decreasing marginal utility will turn out to be crucial for the optimal training schedule. 


Exercise 10 a) 
The following objective function 


max U; = E; {BR (Firm) + EEr 87 *u (e-)} (H.4.1) 


subject to the constraint 
F1 = (1 — ô) Fp + € (H.4.2) 


appears reasonable. 
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Exercise 10 b) 


When we are interested in a training schedule in a deterministic world, we do not need 
to form expectations. 


e DP1: Bellman equation and first-order condition for optimal effort 


The optimal value (value function) of the program is given by V (F;) = max U;. We 
can rewrite the objective function (H.4.1) in the following way 


U, Bh (Fram) + u (er) + DEER 8u (er) 


= ule) +6 [8th (Fim) + S aa T (e,)| = u (er) + BU 41. 


If we assume optimal behaviour and replace U, by the optimal value V (F;), we get 
the Bellman equation V (F;) = max,, {u (e+) + BV (Fi41)}. We calculate the first-order 
condition with respect to er, u’ (e,)+8V’ (Fi41) a = 0. From (H.4.2) we get OF;41/0e; = 
1 which we use for the first-order condition 


u (ex) ot pv" (Fir) =0 8v (ex) = —pV' (Fi) (H.4.3) 


Out of the first-order condition we get the optimal effort given a specific fitness state 
et = Ct (Fi) 


e DP2: Evolution of the costate variable 
We start from the maximized BE by inserting e; (F;) 
V (Fi) = u (ee (Fi)) + BV (C — 6) Fi + e: (2) 
The derivative of with respect to F, is given by (using the envelope theorem) 
V' (Ft) = BV" (Figs) [L — ô] (H.4.4) 


Replacing V’ (F,+1) by the first-order condition (H.4.3) gives 


Perepe m=: (H.4.5) 
p 


Shifting this by one period yields 


V" (Fist) = >u (e141) [1 — Sti]. (H.4.6) 
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e DP3: Inserting first-order conditions 
Inserting (H.4.5) and (H.4.6) into (H.4.4) gives 
=u (ez) [1 — ô] = — pu (e11) [L — Se] [1 — de] > ul (er) = Bul (ee41) [L — be 41] 


u’ (ex) 
> Doh A 
Bu’ (et41) 
That is, relative marginal utilities should equal 1 minus depreciation rate. 
When we look back to the assumption of increasing or decreasing marginal utility we 
see the effect of this assumption on the optimal training schedule by restating (H.4.7) 


=1-S44 (H.4.7) 


u’ (et) 
w’ (€r41) 


= p [1 = t1] 


We see that the right hand side is smaller than one. It must hold, therefore, that u’ (e+) < 
u' (e441). If we make the standard assumption that utility has negative and decreasing 
marginal utility (positive first and negative second derivative), we find ep < e441, i.e. we 
will increase our training effort day by day. 

If the assumed utility function has negative and increasing marginal utility, we have 
e: > €441. Training effort will decrease until t + m. 


Exercise 10 c) 


Let now 6 be stochastic. The objective function is given by (H.4.1) 


max U; = E; {87h (Fram) + UEP 8" *u (e,)} 


and we can reformulate it as we did in b) above. 
e DP1: Bellman Equation and first-order condition 


Again, the optimal value is defined by V (F;) = maxy.,} U; subject to F41 = (1 — ô+) F+ 
e,. Similar to b) we can calculate the Bellman equation as V (F;) = max,, {u (e+) + BEV (Fisi)}. 
First-order condition with respect to e; reads 


OF, 
u (e+) + BEV” (F1) = = 0 
t 


With ore = 1 from (H.4.2) we get 


u' (e1) + BEY’ (Faa) =0 & w (e) = —BE,V' (Fist) 


which implies e; = e: (F;). 
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e DP2: Evolution of the costate variable 
By inserting e; = e: (F;) from DP1 we get the maximized Bellman equation 
V (Fi) = u (ee) + BEV (Fea) 
and Fis, = Faa (Fi, e¢ (F})). Calculating the derivative with respect to F, gives 


OF 41 


v’ (F) = BEV’ (Fiat) OF 
t 


= BEL {V" (Fii) [1 — 54]} 
As we know 6; in t we can put [1 — ô+] in front of the expectation and get 
Vv’ (Fi) = B [1 + ô+] EV” (Fi41) (H.4.8) 


Inserting first-order condition gives 


V'(F) = Bl — 6] B -zv ce} E ale) (H.4.9) 


Shifting one period 
V’ (Fi41) = [1 = Orta u (e141) (H.4.10) 


e DP3: Inserting the first-order condition 
Inserting (H.4.9) and (H.4.10) into (H.4.8) gives 
— [1 — ôi] u (ex) = 8 [1 — ôi] Be {— [L — der] u (er) 


Bu (e1) 1 =a 


1 
u! (e) Móna] 


Su (e) = BE: {[1 — fnil u (ery) } S E: i 


The interpretation is standard and requires relative marginal utilities to be equal to rel- 
ative marginal “cost”. As in all intertemporal conditions, future values are discounted 
either by time-preference or by “interest” rate. 


Appendix I 


Solutions to exercises of chapter 10 


I.1 Differentials of functions of stochastic processes 
II — ex. 3 


Exercise 3 a) 


For F (x,y) = xy and dx = f” (x,y) dt + g” (x,y) dq”, dy = f” (x,y) dt + g” (x, y) dg”, 
using (10.2.9) we know that 


dF (x,y) = lyf" (x,y) + xf” (x,y) dt + {lx + 9° (x,y)] y — cy} dae 
+ {z [y + g” (x, y)| — zy} day. 
Opening the brackets and rearranging we get 
dF (x,y) = [yf" (x,y) + xf” (a, y)] dt +g” (x,y) ydqe + 9” (£, y) zdqy 


= ylf* (x,y) dt + 9° (x,y) dde] + x| f” (x, y) dt + g” (x, y) dq] 
=dx =dy 


ydx + xdy. 
Exercise 3 b) 


For F (x,y) = xy and da = f* (x,y) dt + g (x,y) dz", dy = f” (x,y) dt + g” (x, y) dz”, 
using (10.2.7) we can repeat the same and see that 


dF (x,y) 
= fo + f” (<, y)y + f” (ay) e+ ; [[9” (x,y)? 0 + 2p.y9° (£, y) 9” (x, y) + [9 (x, y)] 0] \ dt 


+ 9° (x,y) ydz” + g” (x, y) zdz” 
= {f° (x,y)y + f” (ey) £ + Payg” (vy) g” (x, y)} dt + 9° (£, y) yd” + g” (£, y) zdz”. 
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Opening the brackets and rearranging we get 


dF (x,y) = ylf? (x,y) dt + g” (x, y) dz*] + a[f* (x,y) dé + g” (x, y) dz*| 


=dxz =dy 
+ Payg” (x,y) 9” (a, y) dt = ydx + zdy + payg” (x,y) g” (x, y) dt. 


Hence, dF (x,y) = zdy + ydx + ppg” (x,y) g” (a, y) dt. This tells us that dF (x,y) = 
xdy + ydx will hold only if p,,, = 0, i.e. when z(t) and y(t) are independent. 


I.2 Correlated jump processes — ex. 4 
Exercise 4 a) 


We are looking for an economic interpretation of labour productivity in sector X 
following dA (t) = adt + Bdqz (t) and labour productivity in sector Y following dB (t) = 
ydt + ddq,(t). Both qx and qy are linear combinations of the q;. As qı appears in both 
processes, the sector specific shocks dA and dB are what could be called “correlated 
Poisson processes”. Note, however, that strictly speaking qs and qy are not Poisson 
processes any more. Shocks can be classified into aggregate shocks to the economy as a 
whole, qı, and idiosyncratic sector specific shocks gz and q3. 


Exercise 4 b) 


We now consider the evolution of GDP in terms of dq, and dq,. GDP is given by 
G (A(t), B (t)) = p,A (t) La +p,B (t) Ly. Now compute the differential of the GDP G (t). 
After having replaced qy and qy in dA and dB 


dA (t) = adt + Bdq, (t) = adt + Bd |q (t) + g (t)], 


dB (t) = ydt + ddqy (t) = ydt + ôd [qı (t) + q3 (d)), 
we apply lemma 10.2.5 to get 
dG (A(t),B(t)) = (a@4+7Gp)dt + [G (4 (t) + 2, B(t) +ô) - G (A (6) , BO) da 
HGA (E) +8, B) -G (40), BW) da 
HG (A(t), B (€) +6) — G (4 (t) , B (€))] das. 


sector specific shock in Y 


Now define as a shortcut 
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With this definition we can rearrange dG (A, B) to get 


dG(A,B) = {ax + yy}dt + {(A+ b) z+ (B +ô)y — (Az + By)} da 
{(A + 8) z + By — (Az + By)} dao 

+{Ax + (B +ô) y -— (Ax + By) } dg 

= {ax + yy} dt + {Bax + dy}dqi + Bxdqz + dydgs 

{ax + yy} dt + Bx (dq, + dqz) + dy (dq, + das) 

= {ax + yy}dt + Bxdq. + dyddy. 


Replacing x and y by p,L, and p,L, respectively yields 


dG (A, B) = {aPrLr + YPy Ly} dt + Bp, Ledqz + py Lydqy. 


I.3 Deriving a budget constraint — ex. 5 
Exercise 5 a) 


The change in the number of shares of stock i held by the household is the change of 
the fraction of savings of the household reserved for the acquisition of the stock 7 divided 
by the price of one share of this stock. 


Exercise 5 b) 
Prices of assets follow 


dv, = aıvıdt + Byvidq, 
duz 


Agvedt + Bov2dqo, 


where qı and q2 are two Poisson processes. We define savings of the household as s = 
Tn + Tong + w — pc. The number of shares of asset i is determined by 


dnı = y—dt, dng = (1 — x) Ê dt. 
Ui U2 
Wealth of the household is defined by a = nivi + novo. 
The budget constraint of the household follows from the differential of a. Using the 
appropriate CVF yields 


S 8 
da = {man + NQV + uix— + v2 (1 — x) 3 dt 
U1 v2 


+ {nq [vi + biv] + neve — (nivi + nev2)} da 
+ {nv + ng [v2 + Bava] — (n1v1 + nava) } dq2 
= f{ayniv1 + aN + s} dt + X2 b;nividqi. 
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Inserting savings and collecting terms gives 


da = {ni la? =] nats oo T| tw- pedt Eh 10; nriVidq. 


Let 0 = == and define r; = œ; + 7+. Then 


da = {aðrı +a (1 — 0) r2 +w — pc} dt + 0abıdqı + (1 — 0) aB,dqo 
{I8 (rı — r2) + r2} a + w — pc} dt + a [68,dqi + (1 — 0) Badqo] . 


I.4 Solving stochastic differential equations — ex. 12 
Exercise 12 a) 


We start from 


de(t) = [a(t) — x(t)] dt + By(t)x(t)dW, (t) + ba(t) dW2(t) 
[—a(t) + a(t)] dt + {Bi (t)x(t) + 0} dW: (t) + {0 + bo(t)} dW2(t). 


Applying (10.4.6), 


Uap = Py Bilt) Bilt) = Br(t), 
Xs = Pi2Bi(t)ba(t), 
gives 
so S ex { f (-1- 58% )) d+ | s)dWy(s o} epee) 
a) = oo [a+ f meme os sf Typ et 


Exercise 12 b) 


To check that the above expression for x(t) is a solution define 


ut) = f (1-501) ds f Bitaw) 


a sd * a(s) — pı2B1(5)b2(8) så * ba(s)dW2(s8) 
A SR | o(s) A J p) 


and get 
AO 2(¢)) =e 2(t). 
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Applying Ito’s Lemma to (y(t), z(t)) we obtain 


1 

dx = e¥z(t) dy(t) + Y%dz(t) + = | eM z(t) (dy(t))? + 2e¥ dy(t)dz(t) 

= 2 | ~ — 
=x(y(t),z(t)) =x(y(t),z(t)) 


1 
= (lOO) [a + 5 (aude | + edz) + May del 
Consider each of the differentials and their products separately: 


dy(t) = (-1 — Bi) dt + B,(t)dW,(t) 


(dy(t))? = B?(t) (dW,(t))? = B2(t)dt 
do(t) = PO PiBr(tbolt) 4 , BONN) 


o(t) olt) 
dy(t)dz(t) = BOOM OMY sa P 


Substituting these back into the expression for dx we get 


de = (olt) ett) WO + 5 D| + LLa + LO dy tae 


=¢(t) =¢(t) 


E E [(-1 7 BH) dt + By (t)dW,(t) + Bilt tat 


Folt) É a(t) — pı2Bı(t)b2(t) | bo(t)dWa(t) pı2Bı(t)b2(t) 
p(t) o(t) p(t) 
= x(y(t), z(t)) [-dt + Bı(t)dAW(¢)] 
+ [a(t) — pı2B1(t)b2(t)] dt + b2(t)dWa(t) + p12B1(t)b2(t) 


dt 


| + e(t) 


= By(t)dWi(t)a(y(t), 2(t)) — (y(t), z(t))dt + a(t)dt + ba(t)dW2(t) 


= a(t) —a(t)]dt + By(t)x(t)dW,(t) + bo(t)dWa(t) = de(t). 


The very last equality shows us that Ito’s Lemma is satisfied, so (I.4.1a)-(1.4.1b) is a 
solution. 
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1.5 Dynamic and intertemporal budget constraints - 
Brownian Motion — ex. 13 
Exercise 13 a) 
The dynamic budget constraint we study is 
da(r) = [r(r)a(r) + w(r) — p(r)e(r)] dt + oa(r)dz(r). 


To solve for a(r), apply the result given in (10.4.6). With A(T) = r(r), EggB(T) = 0? and 
“po(T) = 0, we get (7) = elo ("u)-30°)dut fo e424) The solution to the above differential 
equation becomes a(r) = ¢(r) [a(0) + fj o (s) [w(s) — p(s)c(s)] ds] which implies 


a(0) = a(r) = fos) bole) = pls)els)] ds 


Replacing 0 by t we obtain 


1 


a(t) = a(r)e Je (rw) 30? )du— f; odz(u) -f e7 Se (r) -30°)du- fë odz(u) [w(s) — p(s)e(s)] ds. 
t 


Applying to the latter result the “no-Ponzi-game” condition 


lim la(r)e SZ (r@) 307) du fi odz(u) i= fj 


TCO 


we get 


aft) = fo e Heo bet) tu-$ of) ip(s)e(s) — w(s)] ds, 


Rearranging this equation gives us the final result 
I e ¥(9)n(s)c(s)ds = a(t) + l e™?®w(s)ds, 
t t 


where y(s) = f? (r(u) — 407) du + f? odz(u). 
Exercise 13 b) 


We start from the BC (10.6.2) and the definition of y(r) = fr (s) = lo?ds F 
J odz (T). We write the BC as 


/ e- i r(s)ds ofe 40°ds— f7 TAT) p (T) z (T) d= at f a fe r(s)ds ofr ło?ds— f adz(T) ay (T) dr. 


t t 


Applying expectations on both sides, we get 


E; i ¿= T r(s)ds o fE 402ds— f7 oder) y (T) č (T) dr ~—a +E, if -a e r(s)ds off ło?ds— f7 odz(T)a1y (T) dr. 
t 


t 
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Now focus on the left-hand side. Pulling the expectation inside and making an in- 
dependence assumption (which is actually not needed - without making it, we have the 
covariance term which later would disappear again when pulling the expectations out 
again) gives 


E l e SE r(s)ds ofe ło?ds— f? aT) y (T) é (T) dr 
t 
= l Ee fe r(s)ds F ede ło?ds— 7 odz(T) F p (T) E (T) dr 
t 
= f Ee SE r(s)ds pea lr—t]-olz(r)-20)] F p (T) c (T) dr. 
t 


Now let us focus on the expected term in the middle and let’s pull out the deterministic 
part. This gives 
E,e27 t-t-2le()-2(0)] = e27 (tl Beeler) -2) 


Remember that z (T) — z (t) is Brownian motion with mean zero and variance 7 — t. Then 
—o |z (T) — z(t)] is N (0, 0? [r — t]) and e~?™-*] is lognormally distributed with mean 


and variance given by (7.3.4). Hence, E,e~?#)-*] = e3°°"-4) and we get 


Eyek7lr-tlele(r)-2) — etol-etol- — elr, 
Using Fe J ")4see" rt] = Bye Si r(s)-"ds the left-hand side therefore reads 
Ey iF e7 Si r(s)ds eff 3e°ds— JP ONE) Crise. iar? = f Ee h r(s)—o7ds P p (T)ce(T)dr 
t t 
=" ia e~ Ji r(s)-0°dsy (T)ce(r)dr. 
t 
When we do the same steps on the RHS, we end up with 
E m ek r(s)-a°dsy (T)c(T)drT = a; + E ie e7 Fe (9)-97 ds, (7) dr 
t 


t 


which is the BC we were looking for. 
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Appendix J 


Solution to exercises of chapter 11 


J.1 Optimal saving under Poisson uncertainty with 
two state variables — ex. 1 


Exercise 1 a) 


We view this as a problem with two state variables. Hence, the BE is a generalization 
of (11.1.4), 


pV (a(t) p(t) = max fu (c0) + ŠEV (a(t),p o} (J.1.1) 


noting that there are now two state variables: the budget a(t) and the price p(t) of the 
consumption good as the price is no more constant evolving according to 


dp (t) = p(t) [gdt + oda (8) (J.1.2) 
e DP 1: First-order condition 


The value of the optimal program changes because of changes in wealth due to a 
Poisson process dq or because of price changes dp. So the differential dV (a (t) , p (t)) is 
given by 

dV (a,p) = Va(a,p) [ra +w — pel dt + Vp (a, p) pgdt 
——_—_——_— 
deterministic change 
+[V (1 + 8) a, (1 +0) p) — V (a, p)] dq (J.1.3) 
O R 
stochastic change (jump) 
This inserted in the Bellman equation yields 


pV (a(t), p(t)) = max u (c ()) + Ed (a(t) ,p(t)) 
t dt 


c(t) 


= max {u (c (£)) + Va (a, p) [ra + w — pe] + Vp (a, p) gp 
+A[V ((1 + 8)a,(1+0)p)-—V (a, p)|} (J.1.4) 
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The first-order condition is 
ue (c) = Va (a, p) p (J.1.5) 


So current utility from an additional unit of consumption u, (c) must equal (future) utility 
from an additional unit of wealth V, (a, p) multiplied by the price p of the consumption 
good. 


e DP 2: Evolution of costate variable 
Starting from the maximized Bellman equation 


pV (a,p) = u(c(a))+Va (a, p) [ra + w — pe(a)] + Vp (a, p) gp 
+X[V (1 + Ba, (1+0)p) -V (a,p)], 


the derivative with respect to a is 


pVa(a,p) = Va (a, p) + Vaa (a, p) [ra + w — pe (a)| + Vpa (a, P) gp 
+A [Vz (a, p) [L + B] — Va (a, p)] 


= Vala, p)[ra +w — pe(a)] = (p — r) Va (a, p) 
—Vpa (a, p) gp — À [Va (@, p) [1 + £] — Va (a, p)] (J.1.6) 


where a = (1 + 8) a and p = (1 + o) p. 
The second step is the calculation of the differential of the partial derivative of the 
value function 


dVa (a, p) = {Vaa (a, p) [ra ai w = pc] ag Vap (a, p) gp} dt 
+ [Va ((1 + b)a, (1+) p) — Va (a, p)] dq 


S dVa(a,p) = {lp =r] Va (a, p) — Vpa (a, p) gp 
—A [Va (&, P) (1 + 8) — Va (a, p)] + Vap (a, p) gp} dt 
+ [Va (4, P) — Va (a, p)] dg (J.1.7) 
where Vpa (a, p) gp and Vap (a, p) gp eliminate as Vpa (a, p)) = Vap (a, p). So 
CE? dva (a p) = {lP — r] Va (a, p) — A [Va (GB) (1 + 8) — Va (a, p)]} dt 
+ [Va (a, P) — Va (a, p)] dg (J.1.8) 
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e DP 3: Inserting first-order conditions 


Now we can replace the marginal value by marginal utility from the first-order con- 
dition (J.1.5) using the appropriate CVF formula: Let F (x,y) be a function of the two 
Poisson processes dx = a (x, y) dt+b (x, y) dq and dy = c (x,y) dt+g (x,y) dq. So generally 
the differential of F (x,y) is given by 

dF (x,y) = {Fra (x,y) + Fye(x,y)} dt 
Applying this to our two Poisson processes dV, and dp given by (J.1.8) and (J.1.2) 
yields: 
duc(c) = {p[lo — r] Va (a, p) — A [Va (a, p) (1 + 8) — Va (a, p)]] + Va (a, p) pg} at 
+ [Va (&, P) — Va (a, p)] dq 
= {p[lo +g -— r] Va (a, p) — À [Va (a, P) (1 + 8) — Va (a, p)]]} dt 
1 [Va (a, p) p — Va (a, p) p] dq 


Exercise 1 b) 
This follows the same steps as the derivation of the Keynes-Ramsey rule in (11.1.10). 
Exercise 1 c) 


When either 3 or o or both are zero, this affects the jumps of wealth and the price, 
a = (1+ 8)a and p= (1 + o) p. The derivation is therefore identical to a) only that 3 or 
o or both are zero. 


J.2 Optimal saving under Brownian motion — ex. 2 


Here, dp/p is assumed to be deterministic, i.e. there are only deterministic price changes 
dp(t) = p(t)gdt. The general Keynes-Ramsey rule for marginal utility is given from 
(11.3.4), 


du'(c) = {(p—r)u (c) — u” (c) ca (t) Ba (t)} dt 
+u” (c) Ca (t) Ba (t) dz (t) + u’ (c) dp (t) /p (t) (J.2.1) 
<> du’ (c) = {(p—r)u' (c) — u" (c) ca (t) Ba (t) } dt 


tu" (c) ¢q (t) Ba(t) dz (t) + ul (c) PM) 9 at 
p(t) 


dul (0) = {(p+g—r)w (c) — u" (c) ca (t) Sa (t)}dt 


x 


+u” (c) ca (t) Ba (t)dz (t) 
SEA 


y 
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e Derivation of dc 


Let f (.) be the inverse function for ue, ie. f(u (c)) = c. For the first derivative of 
the inverse function we apply the chain rule and get 


df (u'(c)) _ df (u (©) du' (c) _ | 


dc du’ (c) de 
Hence, 
df (u' (c)) 1 
= J.2.2 
du! (c) Ho ( ) 
1 
= — PA 
7 (J.2.3) 


Now calculate the second derivative of f (u’ (c)) = c with respect to c applying the 
chain rule once again as well as the product rule 


PIWO _ yoru 
dc? 
_ Ëf) dui (dule) | Bul (o) df (o) _ 
dlw (A)? de — de v du’ (c) 
os f(u (e) _ a ao oe d? f (u! (c)) (42.2) u” (c) ats 
d(u'(c))? [ee] 2 d(u' (c)) 2 [ee] 2 
CORRAG T 


aeh [u"(@)]8 
Now applying Ito’s Lemma (see chapter 9.2.2, p. 172) to f (uc) yields 


PU) = fidt fut () dA fay () dz + Z fuut? (at 


= fy + fuex (.) + T or dt + fuy (.) dz 


ORD 1 F [lo +g =r) (e) = u" (o) ca (E) Pat) 
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=e uy! (c) x = u! (c) p fa lu” (c) ; : : 
S des TAI ge T a (t) O+ VA [ca (t) 8 oP ya 
+ca (t) Ba (t) dz 
ou" (c) eee z u! (<) A 1 m (c) : : : 
ar rea l Ee a(t)Balt) +5 To) [ca (t) Ba (t)] bar 
p z = Ca (t) Ba (t) de (J.2.5) 


e Keynes-Ramsey rule for utility function u (c (t)) = Inc (t) 


Given the utility function u(c(t)) = Inc(t), we know that wu’ (ce) = 4, u”(c) = -4 
and u” (c) = 24. Using this in (J.2.5) yields 


Sac = fr +g-p- aa (t) Ba (t) + ; i [ca (t) Ba or} dt 


+500 (t) Ba (t) dz 


“ = fr Ffsp= A (t) 8a (t) + 5 [ca (t) Ba or} dt 
-Že (£) ba (t) de 


J.3 Adjustment cost — ex. 3 


The maximization problem of the firm has the following structure, 


maxi(r))., I = E; ie eT") [pk(T)® — ci(r)?] dr ( ) 
s.t. dk(r) = (ilt) — 6k(7)) dr ( ) 

dc(r) = a.c(r)dt + oeclT)dzelT) (J.3.3) 

dp(T) = app(T)dT + opp(7)dz(r) (J.3.4) 

k(t) = ki, c(t) = c, p(t) = pe. (J.3.5) 


Exercise 3 a) 


Here we assume a, = 0, = 0. Therefore the price is p(T) = ps in each point in 
time r. Now we can solve the maximization problem (J.3.1) - (J.3.5) by using dynamic 
programming. 
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e DP 1: Bellman equation and first-order condition 


In the first step we have to determine the Bellman equation of the maximization 
problem. Therefore we use Ito’s Lemma (10.2.2), thus we yield the Bellman equation 


_ BW (k, za 


rV (k,c) = max for — ci’ 4 TE 


a(t) 


1 
= max l pk“ — ci? + Vp [i — 6k] + Veaec + Vaate} . (J.3.6) 
ut 


The first-order condition according to the Bellman equation (J.3.6) has the form 
Vp = 2ct (J.3.7) 


implying 
Vie = 2i: (J.3.8) 


e DP 2: Evolution of the costate variable 


The derivative of the maximized Bellman equation (J.3.6) with respect to the state 
variable k gives (using the Envelope theorem) 


1 
rV, = a + Vaali — dk) — Ve + Viettce + Z Vreca 2e 


1 
E Ver(i — Ök) + Veotice + 5 Vreca ee” = (r + 8) V; — a a 


(J.3.9) 


Computing the differential of the shadow price of capital V; gives, using Ito’s Lemma 
(10.2.2) 


1 
dV; = Vix (i m ôk) dt + Vke (a.cdt + o-cdz) + 5 Vreer dt 


i (J.3.10) 
= Vix | (i — 0k) + Vkc&ce + g Vreer dt + oecdze 
If we insert (J.3.9) into (J.3.10) we get 
dV, = [(r + ô) Vp — aepk*"] dt + Veco -cdZc (J.3.11) 


e DP 3: Inserting the first-order condition 
Now we replace Vp by (J.3.7) and Vic by (J.3.8) in equation (J.3.11). Therefore we get 
2dci = |(r + ô — a,)2ci — apk*"} dt + 2acidz. 


I ght ss (J.3.12) 
p=, db = |(r+d—a,)b— zpk” dt + oebdze 
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with b = ci. Finally, we get the stochastic differential equation describing the evolution 
of investments 7 by applying Ito’s Lemma (10.2.2) on the function J(b, c) = b/c. Together 
with the stochastic differential equations (J.3.3) and (J.3.12) we get 


1 
di = dJ(b,c) = dy l [e + ô)b — T dt + vate.) 


ë 


+ Je {a-cdt + o-cdz.} 


s” 
-2 
= 928 
1 272 | 7 2 22 
+= | Jæ OO +2 Jie ogbe =| Je ORG: | dt 
2 Ime “YH 
=0 beng all _b 
T eZ ~ e3 
b 1 b b 
= | (r+ 8-0? -— a) — ——apjk’ “| dt —oo-dz.+ o,-dz 
Cc 2 c C 
> 
; Opt 
= ô — o? — OS 
I(r + Oo; — Qc)i Jeki a 


Together with the stochastic differential equations (J.3.2) and (J.3.3) we can completely 
describe the investment behavior 7. 


Exercise 3 b) 
Same principle as in a) only that a. = ce = 0 now. 


Exercise 3 c) 


This is a long derivation but it follows the same principles as above, combining Brownian 
motion with Poisson jumps. 


Exercise 3 d) 


For this exercise we assume a, = ge = 0. Therefore the consumption is c(T) = ¢ in 
each point in time 7. Furthermore we assume that zp is a Poisson process. Now we can 
solve the maximization problem (J.3.1) - (J.3.5) by using dynamic programming. 
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e DP 1: Bellman equation and first-order condition 


In a first step we have to determine the Bellman equation. Therefore we use the CVF 
(10.2.4), thus we yield the Bellman equation 


rV (k, p) = max (J.3.13) 


l pk* — ci? + V; [i — ôk] + Vpapp } 
i(t) 


+A [V (k, (1 + op) p) — V (k, p)] 
As before, the first-order condition of the Bellman equation (J.3.13) has the form 

V; = 2ci. (J.3.14) 
e DP 2: Evolution of the costate variable 


The derivative of the maximized Bellman equation (J.3.13) with respect to the state 
variable k gives (using the Envelope theorem), the equation 
ap 
ki-a 
Computing the differential of the shadow price of capital V; gives, using the CVF (10.2.4) 


dV; = (i — ôk) Vigdt + appVepdt + Vi. = v dq (J.3.16) 


with V, = V (k, p(1 + ap))- 
If we insert (J.3.15) into (J.3.16) we get 


We = {(r +8) Ve- A [An Vil f at + Vi — Vi dq (J.3.17) 


e DP 3: Inserting the first-order condition 


Now we replace V, and V, by (J.3.14) in equation (J.3.17). Therefore we get 


2cdi = { 2e(r +5)i- a — eds — ij} dt + 2c[i — ildq 
eat o ee ee E 
< di = {i(r + 6) r E Alt il} dt + [i — i]dq 


Together with the SDEs (J.3.2) and (J.3.4) we can completely describe the investment 
behavior i. 
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Exercise 4 a) 


What is the optimal static employment level | for a given technological level q? The 


1 
demand function is x = ®p-§ & p= (2) © , the technology is given by x = all, a>1 


and the wage rate is w. 
Profits are then given by 7(I) = al (4) 
respect to employment reads 


i 
= 


— wl and the first-order condition with 


The last ratio simply divides marginal costs by the mark-up parameter. 
Exercise 4 b) 


The intertemporal objective function reads II, = f° e~?!"4r(I(r))dr. Since q is con- 
stant over time, the optimal employment level is constant as well and given by previous 
solution 2 

® (wes) 
Lr w Yr >t. 
al 
The result does therefore not change with respect to (a). 


Exercise 4 c) 


We now determine both / and /, optimally. The objective function now reads 


My = f etn), la(r))dr 
t 
with instantaneous profits being given by 


AUP a) = alr) (eres) — llr) tl] 


and q(T) being a Poisson Process with arrival rate A(l4). 
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e DP 1: Bellman equation and first-order conditions 


The optimal programme is V(q(t)) = maxyi7),,(7); H(t) subject to the constraints. 
The general Bellman equation reads 


1 
PVE) = max, {w(t l(t) + BAVU) 
With the differential dV (q(t)) = {V (q + 1) — V(q)}dq and after forming expectations, 
EdV (q(t) = {V (q + 1) — V(q) s Eda = {V (4 + 1) — V (0) SAU), 
the specific Bellman equation is 


pV (q(t)) = a {m(U(E), lg(t)) +All + 1) -Vol 


The first-order conditions are 
1 w: a7 
3 2 (m) 


aat) 


fon at(e — 1) (aa) —_w=06l(t)= 


E 
w = A,|Vi¢q+1)—-V(q)]. 


’ 


Exercise 4 d) 


We now compute the expected output level for 7 > t. We can write output in 7 as 


a(T) = aq (T) I(T) = aq (7) i (e) =o (: = E) a. 


aq (T) WE 


Its mean is 


Sy -1\f 
wo = aeo en 


; E 
Lemma 8 (section 9.4.1) hi E€— 1 ft) el -DAG (7-8) pi 
wE , , 


where the last step assumed a constant arrival rate. 


End of solutions manual 


For references and index, please see page 299 and onwards. 


